Cloth automatic sorting method and system based on computer vision and deep learning

Abstract

The present application relates to a kind of cloth automatic sorting method and system based on computer vision and deep learning, comprising the following steps: S1: cloth surface original image is obtained by acquisition unit, and synchronous acquisition metadata, including temperature and humidity, calibration reference, timestamp, obtain original image stream and basic metadata;S2: original image stream is preprocessed, and the image after preliminary ROI region is preprocessed is obtained;S3: according to the image after preliminary ROI region is preprocessed, improved Mask R-CNN model is used, detects and divides single cloth, and the coordinate information of each cloth is output;S4: based on multi-branch deep neural network, extract cloth category, pattern, color, material, defect multi-label attribute, obtain cloth attribute feature;S5: according to cloth attribute feature and coordinate information, according to order demand generation sorting priority and specific sorting strategy.The present application effectively improves cloth sorting efficiency and reliability.

Description

Technical Field

[0001] This invention relates, and more particularly, to an automatic fabric sorting method and system based on computer vision and deep learning. Background Technology

[0002] In recent years, with the global manufacturing industry transforming towards "Industry 4.0" and "intelligent manufacturing," the textile industry is facing a critical juncture of upgrading and transformation. Given China's status as the world's largest textile producer, the level of automation in fabric production and sorting directly impacts the efficiency of the supply chain and its international competitiveness. However, at present, textile sorting both domestically and internationally remains primarily manual, with an automation rate of less than 30%. The limitations of manual sorting are increasingly apparent, especially when dealing with multi-category, small-batch, and highly variable fabric sorting tasks. Typical manual sorting processes rely on operator visual inspection and manual classification, which is not only inefficient (averaging only 50-80 pieces of fabric per person per hour) but also suffers from significant subjective judgment discrepancies, difficulty in ensuring consistency, and fatigue from prolonged work, becoming one of the bottlenecks restricting the digital transformation of the textile industry.

[0003] The main technical challenges in the textile fabric sorting field are as follows: First, fabrics themselves are characterized by diverse materials, complex patterns, and rich colors, and are prone to wrinkling and overlapping, increasing the difficulty of visual recognition. Second, there are many types of fabric defects (including more than 20 common defect types such as broken warp, broken weft, oil stains, and color differences), and some defects are extremely subtle, placing extremely high demands on the accuracy of detection algorithms. Third, the attribute labeling system of fabrics is complex, containing multi-dimensional features such as category, pattern, color, and material, requiring the support of a multi-branch deep network architecture. Fourth, sorting decisions need to comprehensively consider order requirements, fabric attribute matching degree, defect level, and spatial location, which is a typical multi-objective optimization problem. Finally, the changes in temperature, humidity, and light in the actual production environment, as well as the randomness of fabric stacking, all pose severe challenges to the environmental adaptability and robustness of the system.

[0004] Currently, domestic and international textile companies generally use ERP systems to manage orders and inventory, but lack intelligent fabric sorting decision support systems. A few semi-automated solutions still require manual intervention in key processes, such as attribute judgment, defect detection, and sorting decisions. With continuously rising labor costs, increasingly stringent customer requirements for delivery time and quality, and the growing popularity of small-batch, multi-category, and customized production models, developing highly intelligent automated fabric sorting systems has become a rigid demand in the industry. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide an automatic fabric sorting method and system based on computer vision and deep learning, which effectively improves fabric sorting efficiency and reliability.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] An automated fabric sorting method based on computer vision and deep learning includes the following steps:

[0008] S1: Acquire the original image of the fabric surface through the acquisition unit, and simultaneously acquire metadata, including temperature and humidity, calibration reference, and timestamp, to obtain the original image stream and basic metadata;

[0009] S2: Preprocess the original image stream to obtain the preprocessed image and the initial ROI region;

[0010] S3: Based on the preprocessed image and the initial ROI region, the improved Mask R-CNN model is used to detect and segment individual pieces of fabric, and output the coordinate information of each piece of fabric.

[0011] S4: Based on a multi-branch deep neural network, extract multi-label attributes such as fabric category, pattern, color, material, and defects to obtain fabric attribute features;

[0012] S5: Based on the fabric attributes and coordinate information, generate sorting priorities and specific sorting strategies according to order requirements.

[0013] Furthermore, the original image stream is preprocessed to obtain the preprocessed image, as follows:

[0014] Based on the current image I, local variance is calculated to obtain the noise level. If the noise is higher than the threshold, a median filter is used; if the noise is lower than the threshold, a Gaussian filter is used to filter it, effectively removing background noise and artifacts.

[0015] Through an adaptive white balance and illumination compensation model, the brightness and color balance of the image are adjusted, and combined with basic metadata, the local and overall brightness and color temperature are dynamically compensated automatically.

[0016] Finally, using edge detection algorithms, the main boundaries of the fabric and background are extracted to generate preliminary candidate ROI regions: using the edge detection output, contour extraction is used to obtain all closed regions or significant edge regions, resulting in preliminary selected candidate ROIs.

[0017] Furthermore, through an adaptive white balance and illumination compensation model, the image's brightness and color are adjusted for balance. Combined with basic metadata, dynamic compensation is automatically applied to local and overall brightness and color temperature, as follows:

[0018] Real-time acquisition of illuminance E and color temperature T measured by the light sensor env When a deviation from the set standard values ​​E0 and T0 is detected, the area requiring compensation is marked.

[0019] To bring the grayscale mean of the RGB three channels of the image back to the same level (improving the physical rationality of the color values) and compensate for color shifts caused by different lighting conditions, the mean value is calculated for each channel c∈{R,G,B}:

[0020] ;

[0021] Let the ideal gray level mean of each channel be . Then the adjustment coefficient is:

[0022] ;

[0023] Corrected pixel values:

[0024] ;

[0025] Automatic compensation for global and local brightness is performed by combining environmental metadata and image brightness distribution;

[0026] Global compensation: Brightness compensation coefficient δ L Depends on ambient illuminance E. If the target illuminance E is 0, and the current illuminance is E, let α be the compensation ratio coefficient:

[0027] ;

[0028] Corrected luminance channel:

[0029] ;

[0030] Local adaptive compensation: The block gain for each block i is g i : ;

[0031] Among them, L i L0 is the average brightness within the block, and L1 is the target brightness. This is the default value;

[0032] Local correction: ;

[0033] Based on the color temperature deviation, dynamically fine-tune each color channel and output a new image after comprehensive correction:

[0034] If the current calculated color temperature is T curr Let the target color temperature be T0 and the color correction gain be γ. c : ;

[0035] Adjust the color temperature sensitive channel:

[0036]

[0037] Where β is the color temperature adjustment coefficient.

[0038] Furthermore, the Mask R-CNN model is improved by including an enhanced backbone network, an optimized feature pyramid network, a dual-head region proposal network, an adaptive ROI alignment module, and an enhanced mask prediction head, as detailed below:

[0039] The enhanced backbone network uses ResNeXt as the base network to extract multi-level features of the image, and embeds channel attention and spatial attention mechanisms into the backbone network.

[0040] Channel attention module: ;

[0041] Where GAP is global average pooling, W1 and W2 are the weights of the fully connected layer, σ is the activation function; ReLU is the activation function; F is the input feature map; M c (F) is the channel attention weight map;

[0042] Spatial attention module: Ms(F) = σ(Conv) 7×7 ([MaxPool(F);AvgPool(F)]));

[0043] Where Ms(F) is the spatial attention weight map; MaxPool(F) and MaxPool(F) represent the maximum pooling and average pooling in the spatial dimension, respectively. Indicates a chain operation; Conv 7×7 This is a 7×7 convolution operation;

[0044] Final attention enhancement feature F′:

[0045] in, Indicates element-wise multiplication;

[0046] The optimized Feature Pyramid Network (FPN) is used to fuse information from different feature layers, improve the model's robustness to scale changes and edge details, and introduces a bidirectional feature flow and texture enhancement module to strengthen the extraction of fabric texture structure and improve the detection capability of small-scale and fine regions.

[0047] Bidirectional feature fusion:

[0048] Standard top-down path: P l =Conv 1×1 (C l )+Upsample(P l+1 );

[0049] Among them, C l P is the feature map of the l-th layer output by the backbone network; lThis is the feature map of the l-th layer from top to bottom; Conv 1×1 It is a 1×1 1×1 convolution; To upsample features to a higher resolution; P l+1 These are features from the previous FPN layer, used for fusion.

[0050] Add a bottom-up path: ;

[0051] in, The features of the l-th layer after bottom-up fusion; Conv 3×3 It is a 3×3 convolution; This is a downsampling operation;

[0052] Texture enhancement module: Add a texture enhancement module for each FPN level: ;

[0053] Among them, T l Texture enhancement features are obtained by dilated convolution (DilatedConv) and regular convolution; DilatedConv3×3 is a dilated convolution.

[0054] Final texture enhancement features: ;

[0055] The dual-head region proposal network adds a texture branch to the standard RPN, which can effectively discover representative fabric anchor boxes and use texture consistency scores to assist in anchor box selection, making the segmented regions more continuous and complete.

[0056] Standard RPN calculation: ;

[0057] Among them, a i It is the i-th anchor frame, o i It is the target score, r i These are the regression coefficients; k is the number of anchor frames;

[0058] Add auxiliary fabric texture branches: ;

[0059] in, It is a texture consistency score, used to evaluate the consistency of texture within the anchor box;

[0060] The adaptive ROI alignment module employs a deformable ROIAlign module and introduces learnable offset parameters.

[0061] f i =AdaptiveROIAlign(F,r i ,Δp)

[0062] Where Δp i, is a learnable offset parameter: Δp i =Conv(ROIAlign(F,r i ));

[0063] The enhanced mask prediction head includes a mask branch and an edge detection branch. The mask branch performs multi-scale fusion, and the edge detection branch enhances the model's ability to perceive fine textures and fabric edges, thereby outputting a segmentation mask that more closely matches the real fabric boundaries.

[0064] ;

[0065] Where s represents the scale; M i The output is a binary probability map, which is used as a segmentation mask; MaskHead s This represents the mask prediction branches at different scales s;

[0066] Where E i It is an edge enhancement feature, obtained through the edge detection branch: E i =EdgeNet(f i );

[0067] EdgeNet represents the edge detection branch.

[0068] Furthermore, the multi-branch deep neural network includes a shared backbone network and dedicated task branches, specifically as follows: The shared backbone network uses ResNeXt-101 as its basic architecture and introduces an attention mechanism to enhance the representation of key features:

[0069] The feature maps extracted from the backbone network are input into four independent but complementary task branches, including category recognition branch, pattern recognition branch, color analysis branch, material characteristics and defect detection branch, to obtain multi-label attributes of fabric category, pattern, color, material and defect respectively.

[0070] Furthermore, based on fabric attribute characteristics and coordinate information, sorting priorities and specific sorting strategies are generated according to order requirements, fabric type, and defect level, as follows:

[0071] Obtain the attribute characteristics, defect type and grade, and spatial coordinate information of each piece of fabric;

[0072] Analyze order requirements and extract specified categories, patterns, colors, materials, acceptable defect types, maximum defect levels, and quantities;

[0073] Calculate similarity for each attribute:

[0074] ;

[0075] Among them, A ikIt is the k-th attribute of the i'-th piece of cloth. It is the k-th attribute of the order requirements, and sim is the similarity function;

[0076] Defect type and grade filtering logic:

[0077] ;

[0078] in, This represents the actual defect type of the i′-th piece of fabric; acceptedDefect is the set of defect types acceptable to the order. represents the actual defect level of the i′-th piece of fabric; maxDefectLevel is the maximum defect level acceptable to the order.

[0079] That is, cloth If the defect type is outside the acceptable range of the order, or the defect level exceeds the maximum tolerance level, then the fabric... Filtered out, deduction of points for defect level:

[0080]

[0081] in, For the i′ piece of fabric, deductions are made for the defect level; Lmax is the maximum acceptable defect level for the order, corresponding to maxDefectLevel; L i′ The actual defect level of the i′i piece of fabric is, i.e. ;

[0082] The sorting priority scoring function is as follows:

[0083] ;

[0084] Where K is the total number of attributes (such as category, pattern, color, material, etc.); λ k The weight of the k-th attribute is μ; μ is the defect penalty coefficient. Priority-related bonuses are awarded; γ represents the spatial score weight.

[0085] Remove fabrics that do not meet the requirements. <A reasonable threshold or it will be eliminated; the remaining fabrics will be processed according to... Sort by highest to lowest

[0086] Select the first Q pieces of fabric in sequence to meet the order quantity;

[0087] By combining the spatial coordinates of the fabric, a sorting order and grasping trajectory are generated, and finally a sorting list is output, with each item containing the fabric ID, attributes, coordinates, priority, and specific operation action.

[0088] Furthermore, based on the spatial coordinates of the fabric, the sorting order and gripping trajectory are generated, as follows:

[0089] Extract the target fabric for this batch from the list of fabrics to be sorted. Each piece of fabric contains its specific spatial coordinates. All coordinate information is collected and verified uniformly through industrial control sensors, machine vision or smart shelves to ensure spatial positioning consistency.

[0090] The path planning module takes the current position of the robotic arm as the starting point, uses the spatial coordinates of all target fabric pieces as node inputs, and adopts the nearest neighbor priority principle. The specific process includes:

[0091] a) At each step, starting from the current position of the robotic arm, find the piece of remaining target fabric that is closest to it;

[0092] b) Select the piece of fabric as the next sorting target and mark it as selected;

[0093] c) Update the current position of the robotic arm to the coordinates of the fabric;

[0094] d) Repeat the above steps until all target fabrics are included in the complete path sequence;

[0095] After obtaining the optimal access order for the fabric, standardized gripping and handling instructions are dynamically generated for each piece of fabric.

[0096] The automatic fabric sorting system based on computer vision and deep learning includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the automatic fabric sorting method based on computer vision and deep learning described above.

[0097] The present invention has the following beneficial effects:

[0098] 1. This invention is based on an improved Mask R-CNN and a multi-branch deep neural network, which organically combines attention mechanism, multi-scale feature fusion, texture enhancement and advanced segmentation methods, significantly improving the accuracy and robustness of fabric detection, segmentation and multi-label attribute recognition. Through channel attention and spatial attention mechanisms, it can effectively enhance the sensitivity to fabric boundaries, texture and minor defects. It can achieve high-accuracy target detection and attribute extraction in complex environments (including changes in lighting, cluttered backgrounds and diverse fabric structures), while supporting real-time processing of large batches of image streams, significantly improving the overall intelligent perception capability of the system.

[0099] 2. This invention supports automatic screening and dynamic sorting of multi-attribute heterogeneous orders through a priority scoring mechanism and multi-parameter comprehensive criteria. It is also deeply integrated with path planning algorithms (such as nearest neighbor priority), which greatly simplifies the deployment and response process of sorting operations and realizes efficient, flexible and traceable automatic sorting decisions.

[0100] 3. This invention can effectively improve sorting efficiency and product consistency, adapt to changing order requirements and flexible production modes, and through systematic data collection and intelligent sorting records, it also provides a rich raw data foundation for subsequent big data analysis, quality traceability and continuous process optimization. Attached Figure Description

[0101] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0102] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0103] refer to Figure 1 In this embodiment, an automatic fabric sorting method based on computer vision and deep learning is provided, including the following steps:

[0104] S1: Acquire the original image of the fabric surface through the acquisition unit, and simultaneously acquire metadata, including temperature and humidity, calibration reference, and timestamp, to obtain the original image stream and basic metadata;

[0105] S2: Preprocess the original image stream to obtain the preprocessed image and the initial ROI region;

[0106] S3: Based on the preprocessed image and the initial ROI region, the improved Mask R-CNN model is used to detect and segment individual pieces of fabric, and output the coordinate information of each piece of fabric.

[0107] S4: Based on a multi-branch deep neural network, extract multi-label attributes such as fabric category, pattern, color, material, and defects to obtain fabric attribute features;

[0108] S5: Based on the fabric attributes and coordinate information, generate sorting priorities and specific sorting strategies according to order requirements.

[0109] In this embodiment, the original image stream is preprocessed to obtain a preprocessed image, as follows:

[0110] Based on the current image I, local variance is calculated to obtain the noise level. If the noise is higher than the threshold, a median filter is used; if the noise is lower than the threshold, a Gaussian filter is used to filter it, effectively removing background noise and artifacts.

[0111] By using an adaptive white balance and illumination compensation model, the brightness and color of the image are adjusted. Combined with basic metadata, the local and overall brightness and color temperature are automatically and dynamically compensated. This compensates for the problems of dark, overexposed or color-shifted images caused by uneven illumination in the acquisition area, effectively enhances the detail expression of the fabric area, and improves the consistency and usability of images under different environments.

[0112] Finally, using edge detection algorithms (such as Canny or Sobel), the main boundaries of the fabric and background are extracted to generate preliminary candidate ROI regions: using the edge detection output, contour extraction (such as findContours in OpenCV) is used to obtain all closed regions or significant edge regions, thus obtaining the preliminary selected candidate ROIs.

[0113] In this embodiment, an adaptive white balance and illumination compensation model is used to adjust the brightness and color balance of the image. Combined with basic metadata, dynamic compensation is automatically applied to local and overall brightness and color temperature, as follows:

[0114] Real-time acquisition of illuminance E and color temperature T measured by the light sensor env When a deviation from the set standard values ​​E0 and T0 is detected, the area requiring compensation is marked.

[0115] To bring the grayscale mean of the RGB three channels of the image back to the same level (improving the physical rationality of the color values) and compensate for color shifts caused by different lighting conditions, the mean value is calculated for each channel c∈{R,G,B}:

[0116] ;

[0117] Let the ideal gray level mean of each channel be . Then the adjustment coefficient is:

[0118] ;

[0119] Corrected pixel values:

[0120] ;

[0121] By combining environmental metadata and image brightness distribution, automatic compensation is applied to global and local brightness; for example, when the lighting is too dark, the overall brightness is increased, and a local gain enhancement algorithm is used for local shadow areas.

[0122] Global compensation: Brightness compensation coefficient δ L Depends on ambient illuminance E. If the target illuminance E is 0, and the current illuminance is E, let α be the compensation ratio coefficient:

[0123] ;

[0124] Corrected luminance channel (e.g., Y channel):

[0125] ;

[0126] Local adaptive compensation: The block gain for each block i is g i : ;

[0127] Among them, L i L0 is the average brightness within the block, and L1 is the target brightness. This is the default value;

[0128] Local correction: ;

[0129] Based on the color temperature deviation, dynamically fine-tune each color channel and output a new image after comprehensive correction:

[0130] If the current calculated color temperature is T curr Let the target color temperature be T0 and the color correction gain be γ. c (For red and blue channels): ;

[0131] Adjust the color temperature sensitive channels (such as red and blue):

[0132]

[0133] Where β is the color temperature adjustment coefficient.

[0134] In this embodiment, the Mask R-CNN model is improved, including an enhanced backbone network (with an attention mechanism), an optimized feature pyramid network (enhanced texture feature extraction), a dual-head region proposal network, an adaptive ROI alignment module, and an enhanced mask prediction head, as detailed below:

[0135] The enhanced backbone network uses ResNeXt as the base network to extract multi-level features of the image, and embeds channel attention and spatial attention mechanisms into the backbone network; enabling the model to focus on key areas such as texture changes and edges, and significantly improving the segmentation performance of different fabrics and complex boundaries.

[0136] Channel Attention Module (CA): ;

[0137] Where GAP is global average pooling, W1 and W2 are the weights of the fully connected layer, σ is the activation function; ReLU is the activation function; F is the input feature map; M c (F) is the channel attention weight map;

[0138] Spatial Attention Module (SA): Ms(F) = σ(Conv) 7×7([MaxPool(F);AvgPool(F)]));

[0139] Where Ms(F) is the spatial attention weight map; MaxPool(F) and MaxPool(F) represent the maximum pooling and average pooling in the spatial dimension, respectively. Indicates a chain operation; Conv 7×7 This is a 7×7 convolution operation;

[0140] Final attention enhancement feature F′:

[0141] in, Indicates element-wise multiplication;

[0142] The optimized Feature Pyramid Network (FPN) is used to fuse information from different feature layers, improve the model's robustness to scale changes and edge details, and introduces a bidirectional feature flow and texture enhancement module to strengthen the extraction of fabric texture structure and improve the detection capability of small-scale and fine regions.

[0143] Bidirectional feature fusion:

[0144] Standard top-down path: P l =Conv 1×1 (C l )+Upsample(P l+1 );

[0145] Among them, C l P is the feature map of the l-th layer output by the backbone network; l This is the feature map of the l-th layer from top to bottom; Conv 1×1 It is a 1×1 1×1 convolution; To upsample features to a higher resolution; P l+1 Features from the next higher level FPN are used for fusion;

[0146] Add a bottom-up path (additional feedback): ;

[0147] in, The features of the l-th layer after bottom-up fusion; Conv 3×3 It is a 3×3 convolution; This is a downsampling operation;

[0148] Texture enhancement module: Add a texture enhancement module for each FPN level: ;

[0149] Among them, T lTexture enhancement features are obtained by dilated convolution (DilatedConv) and regular convolution; DilatedConv3×3 is a dilated convolution used to obtain textures with a large field of view.

[0150] Final texture enhancement features: ;

[0151] The dual-head region proposal network adds a texture branch to the standard RPN, which can effectively discover representative fabric anchor boxes and use texture consistency scores to assist in anchor box selection, making the segmented regions more continuous and complete.

[0152] Standard RPN calculation: ;

[0153] Among them, a i It is the i-th anchor frame, o i It is the target score, r i These are the regression coefficients; k is the number of anchor frames;

[0154] Add auxiliary fabric texture branches: ;

[0155] in, It is a texture consistency score, used to evaluate the consistency of texture within the anchor box;

[0156] The adaptive ROI alignment module employs a deformable ROIAlign module and introduces learnable offset parameters, enabling the model to adaptively capture the true boundaries of the fabric, thereby reducing segmentation errors for complex-shaped fabrics.

[0157] f i =AdaptiveROIAlign(F,r i ,Δp)

[0158] Where Δp i , is a learnable offset parameter: Δp i =Conv(ROIAlign(F,r i ));

[0159] The enhanced Segmentation Head consists of a mask branch and an edge detection branch. The mask branch performs multi-scale fusion, while the edge detection branch enhances the model's ability to perceive fine textures and fabric edges, thereby outputting a segmentation mask that more closely matches the actual fabric boundaries.

[0160] ;

[0161] Where s represents the scale; M i The output is a binary probability map, which is used as a segmentation mask; MaskHeads This represents the mask prediction branches at different scales s;

[0162] Where E i It is an edge enhancement feature, obtained through the edge detection branch: E i =EdgeNet(f i );

[0163] EdgeNet represents the edge detection branch.

[0164] In this embodiment, the multi-branch deep neural network includes a shared backbone network and dedicated task branches, as follows: The shared backbone network uses ResNeXt-101 as its basic architecture and introduces an attention mechanism to enhance the representation of key features:

[0165] Input layer: Receives RGB images of fabric (standard size 224×224 or higher resolution)

[0166] Low-level convolutional group: captures basic features such as edges and textures (5 layers of convolution + pooling)

[0167] Intermediate residual blocks: Extract intermediate semantic features (4 sets of residual blocks, 16 layers in total)

[0168] High-level feature layer: Obtains high-level semantic features (2 sets of residual blocks, 8 layers in total).

[0169] Attention Enhancement: Channel attention (CA) and spatial attention (SA) are introduced at each level to enhance feature representation.

[0170] Feature pyramid: Constructing multi-scale feature representations to adapt to fabric attribute requirements at different granularities;

[0171] The feature maps extracted from the backbone network are input into four independent but complementary task branches, including category recognition branch, pattern recognition branch, color analysis branch, material characteristics and defect detection branch, to obtain multi-label attributes of fabric category, pattern, color, material and defect respectively.

[0172] a. Category Recognition Branch

[0173] Objective: To predict the fiber composition of fabrics (e.g., cotton, polyester, linen, blends, etc.).

[0174] Structure: Global average pooling → 1024-dimensional fully connected layer → ReLU activation → Dropout (0.5) → Fully connected layer (number of classes) → Softmax

[0175] Loss function: Cross-entropy loss (CE Loss)

[0176] Output: Probability distribution for each category, such as [Cotton: 0.92, Polyester: 0.05, Linen: 0.02, Blend: 0.01].

[0177] b. Pattern Recognition Branch

[0178] Objective: To identify the pattern structure of fabrics (such as solid colors, stripes, checks, prints, jacquard, etc.).

[0179] Structure: Global average pooling → 1024-dimensional fully connected layer → ReLU → Dropout (0.4) → Fully connected layer (number of flower shapes) → Softmax / Sigmoid

[0180] Loss function: Use Binary Cross-Entropy (BCE Loss) in multi-label cases.

[0181] Output: Probability of each pattern, such as [solid color: 0.03, stripes: 0.95, checkered: 0.01, printed: 0.05, jacquard: 0.00]

[0182] c. Color Analysis Branch

[0183] Objective: To extract the primary and secondary color information of fabric (supporting multi-color combinations).

[0184] Structure: Global average pooling + color clustering module → 512-dimensional fully connected layer → Multi-label color output layer → Sigmoid

[0185] Loss function: Multi-label binary cross-entropy loss

[0186] Output: Color category and percentage, such as [Blue: 0.65, White: 0.30, Red: 0.05]

[0187] d. Material Properties Branch

[0188] Objective: To evaluate the physical properties of fabrics (such as thickness, softness, elasticity, and luster).

[0189] Structure: Global average pooling → Convolutional layer for special material feature extraction → 512-dimensional fully connected layer → Multi-task output (regression + classification)

[0190] Loss function: Combining mean squared error (MSE) and cross-entropy loss

[0191] Output: Material-related properties such as thickness index (1-10) and softness index (1-10).

[0192] f. Defect detection branch:

[0193] Input data: The network receives a single piece of fabric mask region obtained by an autonomous segmentation module (such as Mask R-CNN); the input is a high-resolution color image of the ROI region on the fabric surface, and synchronous metadata (such as shooting light, time, temperature and humidity) from the original image stream.

[0194] Employing a deep structure based on convolutional neural networks (CNNs), it possesses strong capabilities in spatial feature extraction and local anomaly detection. A multi-task loss mechanism is employed to achieve multi-objective collaborative learning for defect location (segmentation / detection), defect type classification, and defect level assessment.

[0195] The feature fusion module weights and integrates image features from different scales and sampling channels to improve the detection rate of minute and hidden defects.

[0196] Test content:

[0197] Defect type identification: Automatically identifies various typical fabric defects such as color difference, oil stains, damage, broken weft, knots, and impurities.

[0198] Defect level assessment: The defect level is precisely quantified based on the defect area, severity, distribution density, etc. (e.g., Level I, Level II, Level III).

[0199] Defect segmentation and localization: Accurately output the mask or bounding box of each defect on the fabric image, and extract spatial information such as the defect center coordinates and the circumscribed rectangle.

[0200] Output:

[0201] Defect Type Label: The main defect category and subtype for each piece of fabric.

[0202] Defect Level: Can be a scalar score or a graded index.

[0203] Defect spatial information: defect region mask, bounding box, physical length / area and other geometric parameters.

[0204] In this embodiment, based on fabric attribute characteristics and coordinate information, sorting priorities and specific sorting strategies are generated according to order requirements, fabric type, and defect level, as follows:

[0205] Obtain the attribute characteristics (category, pattern, color, material), defect type and grade, and spatial coordinate information of each piece of fabric;

[0206] Analyze order requirements and extract specified categories, patterns, colors, materials, acceptable defect types, maximum defect levels, and quantities;

[0207] Calculate similarity for each attribute:

[0208] ;

[0209] Among them, A ik It is the k-th attribute of the i'-th piece of cloth. It is the k-th attribute of the order requirements, and sim is the similarity function;

[0210] Defect type and grade filtering logic:

[0211] ;

[0212] in, This represents the actual defect type of the i′-th piece of fabric; acceptedDefect is the set of defect types acceptable to the order. represents the actual defect level of the i′-th piece of fabric; maxDefectLevel is the maximum defect level acceptable to the order.

[0213] That is, cloth If the defect type is outside the acceptable range of the order, or the defect level exceeds the maximum tolerance level, then the fabric... Filtered out, deduction of points for defect level:

[0214]

[0215] in, For the i′ piece of fabric, deductions are made for the defect level; Lmax is the maximum acceptable defect level for the order, corresponding to maxDefectLevel; L i′ The actual defect level of the i′i piece of fabric is, i.e. ;

[0216] The sorting priority scoring function is as follows:

[0217] ;

[0218] Where K is the total number of attributes (such as category, pattern, color, material, etc.); λ k The weight of the k-th attribute (set according to business priority; for example, if color requirements are high, then λ). color (Large); μ is the defect penalty coefficient; Priority-related scores are added (e.g., first-in-first-out, time weight, spatial distance, etc.); γ is the spatial score weight.

[0219] Remove fabrics that do not meet the requirements. <A reasonable threshold or it will be eliminated; the remaining fabrics will be processed according to... Sort by highest to lowest

[0220] Select the first Q pieces of fabric in sequence to meet the order quantity;

[0221] By combining the spatial coordinates of the fabric, a sorting order and grasping trajectory are generated, and finally a sorting list is output, with each item containing the fabric ID, attributes, coordinates, priority, and specific operation action.

[0222] In this embodiment, the sorting order and gripping trajectory are generated based on the combined spatial coordinates of the fabric, as follows:

[0223] Extract the target fabric for this batch from the list of fabrics to be sorted. Each piece of fabric contains its specific spatial coordinates. All coordinate information is collected and verified uniformly through industrial control sensors, machine vision or smart shelves to ensure spatial positioning consistency.

[0224] The path planning module takes the current position of the robotic arm as the starting point, uses the spatial coordinates of all target fabric pieces as node inputs, and adopts the nearest neighbor algorithm. The specific process includes:

[0225] a) At each step, starting from the current position of the robotic arm, find the piece of remaining target fabric that is closest to it;

[0226] b) Select the piece of fabric as the next sorting target and mark it as selected;

[0227] c) Update the current position of the robotic arm to the coordinates of the fabric;

[0228] d) Repeat the above steps until all target fabrics are included in the complete path sequence;

[0229] After obtaining the optimal access order for the fabric, standardized gripping and handling instructions are dynamically generated for each piece of fabric, specifically including:

[0230] a) Move to target coordinates: The robotic arm moves sequentially to the precise spatial position of the next piece of fabric according to the path;

[0231] b) Positioning adjustment: If necessary, fine-tune the attitude of the end effector to match the pick-up and drop-off direction of the fabric;

[0232] c) Grip descent: The robotic arm controls the gripper or suction cup to descend, approach and contact the fabric surface;

[0233] d) Complete the gripping: Perform actions such as clamping, sucking, or wrapping to firmly grasp the fabric;

[0234] e) Lifting and transporting: After gripping, the robotic arm is safely lifted and moved to the designated collection area or sorting table according to the transport path;

[0235] f) Fabric release: Execute clamp opening or suction cup disconnection in the collection area to safely release the fabric;

[0236] g) Update status: Record that the sorting operation has been completed and the machine is ready to move to the next target point.

[0237] All actions are standardized into a sequence of instructions that the equipment can recognize, and can be flexibly adjusted according to the actual situation on site, such as adding obstacle avoidance steps or buffer time.

[0238] The automatic fabric sorting system based on computer vision and deep learning includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the automatic fabric sorting method based on computer vision and deep learning described above.

[0239] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0240] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0241] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0242] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0243] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An automatic fabric sorting method based on computer vision and deep learning, characterized in that, Includes the following steps: S1: Acquire the original image of the fabric surface through the acquisition unit, and simultaneously acquire metadata to obtain the original image stream and basic metadata; S2: Preprocess the original image stream to obtain the preprocessed image and the initial ROI region; S3: Based on the preprocessed image and the initial ROI region, the improved Mask R-CNN model is used to detect and segment individual pieces of fabric and output the coordinate information of each piece of fabric. S4: Based on a multi-branch deep neural network, extract multi-label attributes of the fabric and obtain fabric attribute features; S5: Based on the fabric attribute features and coordinate information, generate sorting priorities and specific sorting strategies according to order requirements; The preprocessing of the original image stream to obtain the preprocessed image is as follows: The image is analyzed by calculating local variance to determine the noise level. If the noise level is higher than the threshold, a median filter is used; if the noise level is lower than the threshold, a Gaussian filter is used to effectively remove background noise and artifacts. An adaptive white balance and illumination compensation model is used to adjust the brightness and color of the image. Combined with basic metadata, dynamic compensation is automatically applied to local and overall brightness and color temperature. Finally, an edge detection algorithm is used to extract the main boundaries of the fabric and background, generating preliminary candidate ROI regions. Using the edge detection output, contour extraction is employed to obtain all closed regions or significant edge regions, resulting in the preliminary selected candidate ROIs. Through an adaptive white balance and illumination compensation model, the brightness and color of the image are adjusted. Combined with basic metadata, dynamic compensation is automatically applied to local and overall brightness and color temperature. Specifically, the illuminance E and color temperature T measured by the light sensor are acquired in real time. env When deviations from the set standard values ​​E0 and T0 are detected, the area requiring compensation is marked; the grayscale mean of the three RGB channels of the image is brought back to the same level to compensate for color shifts caused by different lighting conditions. For each channel c∈{R,G,B}, the mean is calculated: ； Where N is the total number of pixels, I c (x,y) is the pixel intensity value of color channel c at pixel coordinates (x,y); Let the ideal gray level mean of each channel be . , Given the mean values ​​of the red, green, and blue color channels respectively, the adjustment factor is: ； Corrected pixel values: ； Automatic compensation for global and local brightness is performed by combining environmental metadata and image brightness distribution; Global compensation: Luminance compensation coefficient δ L Depends on ambient illuminance E. If the target illuminance E is 0, and the current illuminance is E, let α be the compensation ratio coefficient: d L =α(E0-E); Corrected luminance channel: ； Local adaptive compensation: The block gain for each block i is g i : ; Among them, L i L0 represents the average brightness within the block, and L0 represents the target brightness. This is the default value; Local correction: ; Based on the color temperature deviation, dynamically fine-tune each color channel and output a new image after comprehensive correction: If the current calculated color temperature is T curr Let the target color temperature be T0 and the color correction gain be γ. c γ c =1+β(T0-T curr ); Adjust the color temperature sensitive channel: ； Where β is the color temperature adjustment coefficient; The improved Mask R-CNN model includes an enhanced backbone network, an optimized feature pyramid network, a dual-head region proposal network, an adaptive ROI alignment module, and an enhanced mask prediction head, as detailed below: The enhanced backbone network uses ResNeXt as the base network to extract multi-level features of the image, and embeds channel attention and spatial attention mechanisms into the backbone network. Channel attention module: M c (F)=σ(W2·ReLU(W1·GAP(F))); Where GAP is global average pooling, W1 and W2 are the weights of the fully connected layer, σ is the activation function; ReLU is the activation function; F is the input feature map; M c (F) is the channel attention weight map; Spatial attention module: Ms(F) = σ(Conv) 7×7 ([MaxPool(F);AvgPool(F)])); Where Ms(F) is the spatial attention weight map; MaxPool(F) and AvgPool(F) represent max pooling and average pooling in the spatial dimension, respectively; [·;·] represents concatenation operation; Conv 7×7 This is a 7×7 convolution operation; Final attention enhancement feature F′: F′=F Mc(F) Ms(F) in, Indicates element-wise multiplication; The optimized Feature Pyramid Network (FPN) is used to fuse information from different feature layers, and introduces a bidirectional feature flow and texture enhancement module. Bidirectional feature fusion: Standard top-down path: P l =Conv 1×1 (C l )+Upsample(P l+1 ); Among them, C l P is the feature map of the l-th layer output by the backbone network; l This represents the l-th layer feature map of the top-down path; Conv 1×1 For 1×1 convolution; Upsample(·) upsamples features to a higher resolution; P l+1 These are features from the previous layer's FPN, used for fusion. Add a bottom-up path: ; in, The features of the l-th layer after bottom-up fusion; Conv 3×3 It is a 3×3 convolution; Downsample(·) is a downsampling operation; Texture enhancement module: Add a texture enhancement module for each FPN level: ; Among them, T l Texture enhancement features are obtained by dilated convolution (DilatedConv) and regular convolution; DilatedConv3×3 is a dilated convolution. Final texture enhancement features: ; The dual-head region proposal network adds a texture branch to the standard RPN, which can effectively discover representative fabric anchor boxes and use texture consistency scores to assist in anchor box selection. Standard RPN calculation: ; Among them, a i It is the i-th anchor frame, o i It is the target score, r i These are the regression coefficients; k is the number of anchor frames; Add auxiliary fabric texture branches: ; in, It is a texture consistency score, used to evaluate the consistency of texture within the anchor box; The adaptive ROI alignment module employs a deformable ROIAlign module and introduces learnable offset parameters. f i =AdaptiveROIAlign(F,r i ,Δp) Where Δp i , is a learnable offset parameter: Δp i =Conv(ROIAlign(F,r i )); The enhanced mask prediction head includes a mask branch and an edge detection branch. The mask branch performs multi-scale fusion, and the edge detection branch enhances the model's ability to perceive fine textures and fabric edges, thereby outputting a segmentation mask that more closely matches the real fabric boundaries. ； Where s represents the scale; M i The output is a binary probability map, which serves as a segmentation mask; MaskHead s This represents the mask prediction branches at different scales s; Where E i It is an edge enhancement feature, obtained through the edge detection branch: E i =EdgeNet(f i ); Here, EdgeNet represents the edge detection branch.

2. The automatic fabric sorting method based on computer vision and deep learning according to claim 1, characterized in that, The multi-branch deep neural network includes a shared backbone network and dedicated task branches, as follows: The shared backbone network uses ResNeXt-101 as its basic architecture and introduces an attention mechanism to enhance the representation of key features: The feature maps extracted from the backbone network are input into four independent but complementary task branches, including category recognition branch, pattern recognition branch, color analysis branch, material characteristics and defect detection branch, to obtain multi-label attributes of fabric category, pattern, color, material and defect respectively.

3. The automatic fabric sorting method based on computer vision and deep learning according to claim 1, characterized in that, Based on fabric attributes and coordinate information, sorting priorities and specific sorting strategies are generated according to order requirements, fabric type, and defect level, as follows: Obtain the attribute characteristics, defect type and grade, and spatial coordinate information of each piece of fabric; Analyze order requirements and extract specified categories, patterns, colors, materials, acceptable defect types, maximum defect levels, and quantities; Calculate similarity for each attribute: ； Among them, A ik It is the k-th attribute of the i'-th piece of cloth. It is the k-th attribute of the order requirements, and sim is the similarity function; Defect type and grade filtering logic: ； in, This represents the actual defect type of the i′-th piece of fabric; acceptedDefect is the set of defect types acceptable to the order. represents the actual defect level of the i′-th piece of fabric; maxDefectLevel is the maximum defect level acceptable to the order. That is, cloth If the defect type is outside the acceptable range of the order, or the defect level exceeds the maximum tolerance level, then the fabric... Filtered out, deduction of points for defect level: ； in, For the i′ piece of fabric, deductions are made for the defect level; Lmax is the maximum acceptable defect level for the order, corresponding to maxDefectLevel; L i′ The actual defect level of the i′-th piece of fabric is, i.e. ; The sorting priority scoring function is as follows: ； Where K is the total number of attributes; λ k Let μ be the weight of the k-th attribute; μ is the defect penalty coefficient. Priority-related bonuses are awarded; γ represents the spatial score weight. Remove fabrics that do not meet the requirements. <A reasonable threshold or it will be eliminated; the remaining fabrics will be processed according to... Sort by highest to lowest Select the first Q pieces of fabric in sequence to meet the order quantity; By combining the spatial coordinates of the fabric, a sorting order and grasping trajectory are generated, and finally a sorting list is output, with each item containing the fabric ID, attributes, coordinates, priority, and specific operation action.

4. The automatic fabric sorting method based on computer vision and deep learning according to claim 3, characterized in that, Based on the spatial coordinates of the fabric, the sorting order and gripping trajectory are generated as follows: Extract the target fabric for this batch from the list of fabrics to be sorted. Each piece of fabric contains its specific spatial coordinates. All coordinate information is collected and verified uniformly through industrial control sensors, machine vision or smart shelves to ensure spatial positioning consistency. The path planning module takes the current position of the robotic arm as the starting point, uses the spatial coordinates of all target fabric pieces as node inputs, and adopts the nearest neighbor priority principle. The specific process includes: a) At each step, starting from the current position of the robotic arm, find the piece of remaining target fabric that is closest to it; b) Select the piece of fabric as the next sorting target and mark it as selected; c) Update the current position of the robotic arm to the coordinates of the fabric; d) Repeat the above steps until all target fabrics are included in the complete path sequence; After obtaining the optimal access order for the fabric, standardized gripping and handling instructions are dynamically generated for each piece of fabric.

5. An automated fabric sorting system based on computer vision and deep learning, characterized in that, It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the automatic fabric sorting method based on computer vision and deep learning as described in any one of claims 1-4.

Images