Fabric defect real-time detection method and system fusing machine vision and deep learning

Abstract

The present application relates to the technical field of fabric defect detection, and particularly relates to a fabric defect real-time detection method fusing machine vision and deep learning. The method comprises the following steps: collecting and registering visible light and near-infrared band images of the fabric, constructing spectral fusion features by calculating spectral response differences and fusing; adopting a two-stage detection strategy, first globally scanning and positioning abnormal areas, and then adaptively adjusting the receptive field to correct the defect boundary; finally, extracting the geometric and gray attributes of the defect area, and matching the classification rule library to determine the defect category. The present application realizes high-precision, adaptive real-time detection and classification of fabric defects.

Description

Technical Field

[0001] This invention relates to the field of fabric defect detection technology, and in particular to a real-time fabric defect detection method and system that integrates machine vision and deep learning. Background Technology

[0002] In the textile industry, automated detection of fabric defects is a crucial step in ensuring product quality. Current conventional practices primarily rely on automated inspection systems based on machine vision. These systems typically use high-resolution industrial cameras to acquire images of the fabric surface in the visible light band, and then use image processing algorithms to analyze the acquired images to identify various defects such as broken warp threads, broken weft threads, stains, and holes.

[0003] Conventional machine vision inspection methods typically focus on analyzing images within a single spectral band (mainly visible light). These methods extract defect features by designing specific image filters, edge detection operators, or texture analysis algorithms. For example, some methods use grayscale statistics, Fourier transforms, or Gabor filters to capture anomalous variations in fabric texture. Other methods attempt to locate discrepancies by comparing the image to be inspected with a defect-free standard template through background modeling or template matching. These solutions, based on traditional image processing, can achieve defect identification to a certain extent.

[0004] However, these conventional methods have significant limitations. The diversity of fabric materials and the complexity of their textures mean that some defects lack sufficient contrast with the background under a single visible light band, making feature extraction difficult. For example, some stains with colors similar to the fabric background, or certain chemical contaminants that only appear under specific spectra, may not be clearly captured in conventional visible light images. This directly affects the sensitivity and reliability of the detection system, easily leading to missed detections.

[0005] Furthermore, conventional methods also have shortcomings in terms of accuracy and adaptability in defect localization. Many algorithms employ fixed detection scales or receptive fields, making it difficult to effectively handle defects with vastly different sizes simultaneously. For large areas of stains and tiny yarn breaks, a fixed analysis window may not be able to balance comprehensive detection with precise boundary conditions. When fine-grained defect classification is required, relying solely on the geometric or grayscale features of visible light images is often insufficient, leading to complex construction of classification rule bases with limited generalization ability, making it difficult to cope with new products or new defect types. Summary of the Invention

[0006] This invention provides a method and system for real-time detection of fabric defects that integrates machine vision and deep learning, which can solve the problems in the prior art.

[0007] A first aspect of the present invention provides a real-time fabric defect detection method integrating machine vision and deep learning, comprising: Multispectral image data of the fabric surface is acquired, including visible light band images and near-infrared band images. The visible light band images and near-infrared band images are registered and aligned to obtain an aligned multispectral image. The multi-band images are subjected to spectral feature fusion. The difference in spectral response between the visible light band image and the near-infrared band image is calculated pixel by pixel to construct a spectral difference feature map. The spectral difference feature map is then concatenated with the visible light band image, and the concatenated multi-channel data is mapped to spectral fusion features through dimensionality reduction projection. Defect localization is performed on the spectral fusion features based on a scale-adaptive cascaded detection strategy. The first-level detection uses a large receptive field detection network to perform a global scan of the spectral fusion features, identify regions with abnormal feature responses, and output an initial defect localization box. The second-level detection dynamically adjusts the receptive field range according to the size of the initial defect localization box to obtain adaptively corrected defect boundary coordinate information. Defect attributes are extracted from the image region corresponding to the defect boundary coordinate information. The area, aspect ratio, shape complexity, and grayscale distribution characteristics of the defect region are calculated. The defect attributes are matched with a preset defect classification rule library to determine the defect category.

[0008] The multi-band images are subjected to spectral feature fusion. A spectral difference feature map is constructed by calculating the spectral response difference between the visible light band image and the near-infrared band image pixel-by-pixel. The spectral difference feature map is then concatenated with the visible light band image, and the concatenated multi-channel data is mapped to spectral fusion features through dimensionality reduction projection. This includes: The light intensity values ​​of each pixel in the visible light band image are extracted as visible light response components, and the light intensity values ​​of the corresponding pixel in the near-infrared band image are extracted as near-infrared response components. The visible light response components and the near-infrared response components are mapped to a unified numerical range, and the numerical difference between the visible light response components and the near-infrared response components is calculated. The numerical difference is used as the spectral difference response value of the corresponding pixel location. The spectral difference response values ​​of all pixel locations are organized into a two-dimensional matrix structure to form the spectral difference feature map. The spectral difference feature map is used as a new channel and concatenated with the original channels of the visible light band image to obtain an extended channel image. The extended channel image is then subjected to convolutional dimensionality reduction operation to compress the number of channels of the extended channel image to the target number of channels, thus obtaining the spectral fusion feature.

[0009] Defect localization based on the scale-adaptive cascaded detection strategy for the spectral fusion features includes: The spectral fusion features are subjected to spatiotemporal domain difference enhancement processing. By constructing a multi-directional gradient operator, the first derivative of the spectral fusion features is calculated in the horizontal, vertical, and diagonal directions to obtain gradient response maps in four directions. The sum of squares of the magnitudes of the gradient response maps in four directions is then taken to generate a comprehensive gradient intensity feature map. The first-level detection uses a sliding window mechanism to perform a global scan on the comprehensive gradient intensity feature map. A multi-scale sliding window combination is set, including three window sizes: large, medium, and small. Each window size scans the feature map with a fixed step size. The mean and standard deviation of the gradient intensity values ​​in each window region are calculated. Candidate windows with abnormal gradient response are selected based on the joint discrimination condition of the mean and standard deviation of the gradient intensity. The spatial coordinates and size parameters of the candidate windows are used as the initial defect location box. The second-level detection refines and extracts regional features from the initial defect location box. Based on the coordinates of the initial defect location box, the corresponding region of interest feature block is cropped from the spectral fusion features. Bilateral filtering is applied to the region of interest feature block to perform edge smoothing. Foreground and background separation is performed on the smoothed feature block using an adaptive threshold segmentation algorithm. Connected components of the foreground region are extracted, and the geometric center and boundary contour of each connected component are calculated. A minimum bounding rectangle is fitted based on the boundary contour, and the coordinates of the four vertices of the minimum bounding rectangle are used as the adaptively corrected defect boundary coordinate information.

[0010] An adaptive threshold segmentation algorithm is used to separate the foreground and background of the smoothed feature blocks. Connected components of the foreground region are extracted, and the geometric center and boundary contour of each connected component are calculated. A minimum bounding rectangle is fitted based on the boundary contour, and the coordinates of the four vertices of the minimum bounding rectangle are used as the adaptively corrected defect boundary coordinate information, including: An adaptive segmentation threshold is dynamically determined based on the global and local grayscale statistical characteristics of the smoothed feature block. The feature block is then subjected to pixel-level binarization using the adaptive segmentation threshold. Pixels with grayscale values ​​higher than the adaptive segmentation threshold are marked as foreground pixels, and pixels with grayscale values ​​lower than the adaptive segmentation threshold are marked as background pixels. Connectivity analysis is performed on the binarized feature blocks. An eight-connected neighborhood search method is used to traverse all foreground pixels, and adjacent foreground pixels are aggregated into the same connected component. The number of pixels contained in each connected component is counted and connected components are filtered. Valid connected components with a number of pixels exceeding the minimum area threshold are retained. The weighted average of the coordinates of all pixels in each valid connected component is calculated to obtain the geometric center coordinates. Boundary tracking is performed on each valid connected component to extract the outer contour point sequence. The minimum bounding rectangle is fitted to the sequence of outer contour points. The optimal rotation angle of the rectangle is determined by calculating the projection range of the contour points at different angles. The extreme values ​​of the horizontal and vertical coordinates of the contour point set are calculated at the optimal rotation angle. The coordinates of the four vertices of the minimum bounding rectangle are constructed based on the extreme values ​​of the coordinates. The coordinates of the four vertices are mapped back to the original image coordinate system through inverse rotation transformation, which serves as the adaptively corrected defect boundary coordinate information.

[0011] Defect attributes are extracted from the image region corresponding to the defect boundary coordinate information, and the area, aspect ratio, shape complexity, and grayscale distribution characteristics of the defect region are calculated, including: The spatial range of the defect region is determined based on the defect boundary coordinate information. In the visible light band image, the geometric center of the defect boundary coordinates is used as the seed point. The region growing algorithm is used to adaptively expand the defect region. The growth criterion is set as the gray difference between adjacent pixels is less than the dynamic growth threshold. The dynamic growth threshold is updated in real time according to the gray variance of the grown region. The complete defect connected region is obtained through the iterative growth process. The boundary of the defect connected region adaptively fits the true shape of the defect. Multi-scale morphological analysis is performed on the defect-connected region to calculate the zero-order moment and the first-order moment of the defect-connected region. The defect area is obtained through the zero-order moment, and the centroid coordinates of the defect are calculated through the first-order moment. Polar coordinate transformation is performed on the defect-connected region with the centroid as the center. The distance sequence from the defect boundary point to the centroid is calculated in the polar coordinate system. The maximum and minimum values ​​of the distance sequence are extracted as the maximum and minimum radii of the defect. The aspect ratio is obtained by calculating the ratio of the maximum and minimum radii. The second-order central moment of the defect-connected region is calculated. A covariance matrix is ​​constructed based on the second-order central moment, and its eigenvalues ​​are calculated. The anisotropy of the defect is characterized by the ratio of eigenvalues ​​as shape complexity. Multi-scale wavelet decomposition is performed on the gray values ​​in the defect-connected region to extract the energy distribution of wavelet coefficients at different scales and directions. The energy distribution vector is used as a representation of the gray-scale distribution characteristics.

[0012] The defect category is determined by matching the defect attributes with a preset defect classification rule base, including: A defect classification and matching mechanism based on fuzzy membership degree is established. The defect area, aspect ratio, shape complexity, and grayscale distribution characteristics are organized into defect feature descriptors. A fuzzy membership degree function is defined for each defect category in the defect classification rule base. The fuzzy membership degree function describes the degree of belonging of the feature parameters to the category. The fuzzy membership degree value of each dimension component of the defect feature descriptor to each category is calculated. The membership degree values ​​of each dimension are fused and calculated through fuzzy inference rules. Calculate the comprehensive membership score of the defect feature descriptor for each defect category, select the defect category with the highest comprehensive membership score as the final classification result, and when the highest comprehensive membership score is lower than the confidence limit, mark the defect as a fuzzy category defect and trigger the manual review process.

[0013] A second aspect of the present invention provides a real-time fabric defect detection system integrating machine vision and deep learning, comprising: An image acquisition unit is used to acquire multispectral image data of the fabric surface. The multispectral image data includes visible light band images and near-infrared band images. The visible light band images and near-infrared band images are registered and aligned to obtain an aligned multispectral image. The spectral feature unit is used to perform spectral feature fusion on the multi-band image. It constructs a spectral difference feature map by calculating the spectral response difference between the visible light band image and the near-infrared band image pixel by pixel. The spectral difference feature map is channel-cascaded with the visible light band image, and the cascaded multi-channel data is mapped to spectral fusion features by dimension reduction projection. The defect localization unit is used to locate defects in the spectral fusion features based on a scale-adaptive cascaded detection strategy. The first-level detection uses a large receptive field detection network to perform a global scan of the spectral fusion features, identify regions with abnormal feature responses, and output an initial defect localization box. The second-level detection dynamically adjusts the receptive field range according to the size of the initial defect localization box to obtain adaptively corrected defect boundary coordinate information. The defect attribute unit is used to extract defect attributes from the image region corresponding to the defect boundary coordinate information, calculate the area, aspect ratio, shape complexity and grayscale distribution characteristics of the defect region, and match the defect attributes with a preset defect classification rule library to determine the defect category.

[0014] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0016] This method significantly improves the accuracy and robustness of fabric defect detection through the acquisition and fusion of multispectral image data. Registration and alignment of visible and near-infrared images ensure spatial consistency of different spectral information, providing a precise data foundation for subsequent analysis. By calculating spectral response differences pixel-by-pixel and constructing spectral difference feature maps, subtle differences between defect areas and normal fabric backgrounds under specific spectra can be effectively amplified. Channel cascading and dimensionality-reduced projection of the difference features with visible light images achieves complementarity and efficient compression of multi-source information, enhancing the characterization ability of the features and making subsequent detection more immune to changes in illumination and fabric texture interference.

[0017] A scale-adaptive cascaded detection strategy was adopted to achieve coarse-to-fine defect localization. The first-stage global scanning with a large receptive field quickly identifies potential abnormal areas, avoiding missed detections and improving detection efficiency. The second stage dynamically adjusts the receptive field based on the initial localization box, which can finely correct defect boundaries and accurately distinguish closely adjacent or multiple small defects, improving localization accuracy. This two-stage collaborative mechanism effectively balances detection speed and localization accuracy, and is particularly suitable for identifying small or ambiguous defects in complex backgrounds.

[0018] Based on precisely located coordinate information, various defect attributes, including geometric shape and grayscale distribution characteristics, are extracted to construct a comprehensive defect descriptor. By matching the extracted attributes with a pre-defined classification rule base, automated and standardized identification of defect types can be achieved. This method can not only identify the presence of defects but also accurately distinguish different categories such as stains, warp breaks, and weft gaps, providing a direct and reliable basis for production quality assessment and process adjustment. The entire process realizes a closed loop from image acquisition, feature enhancement, precise positioning to intelligent classification, meeting the urgent need for real-time, high-precision fabric defect detection in industrial settings. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a real-time fabric defect detection method that integrates machine vision and deep learning. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0022] Figure 1 This is a flowchart illustrating the real-time fabric defect detection method integrating machine vision and deep learning according to an embodiment of the present invention. Figure 1 As shown, the real-time fabric defect detection method integrating machine vision and deep learning includes: Multispectral image data of the fabric surface is acquired, including visible light band images and near-infrared band images. The visible light band images and near-infrared band images are registered and aligned to obtain an aligned multispectral image. The multi-band images are subjected to spectral feature fusion. The difference in spectral response between the visible light band image and the near-infrared band image is calculated pixel by pixel to construct a spectral difference feature map. The spectral difference feature map is then concatenated with the visible light band image, and the concatenated multi-channel data is mapped to spectral fusion features through dimensionality reduction projection. Defect localization is performed on the spectral fusion features based on a scale-adaptive cascaded detection strategy. The first-level detection uses a large receptive field detection network to perform a global scan of the spectral fusion features, identify regions with abnormal feature responses, and output an initial defect localization box. The second-level detection dynamically adjusts the receptive field range according to the size of the initial defect localization box to obtain adaptively corrected defect boundary coordinate information. Defect attributes are extracted from the image region corresponding to the defect boundary coordinate information. The area, aspect ratio, shape complexity, and grayscale distribution characteristics of the defect region are calculated. The defect attributes are matched with a preset defect classification rule library to determine the defect category.

[0023] The multi-band images are subjected to spectral feature fusion. A spectral difference feature map is constructed by calculating the spectral response difference between the visible light band image and the near-infrared band image pixel-by-pixel. The spectral difference feature map is then concatenated with the visible light band image, and the concatenated multi-channel data is mapped to spectral fusion features through dimensionality reduction projection. This includes: The light intensity values ​​of each pixel in the visible light band image are extracted as visible light response components, and the light intensity values ​​of the corresponding pixel in the near-infrared band image are extracted as near-infrared response components. The visible light response components and the near-infrared response components are mapped to a unified numerical range, and the numerical difference between the visible light response components and the near-infrared response components is calculated. The numerical difference is used as the spectral difference response value of the corresponding pixel location. The spectral difference response values ​​of all pixel locations are organized into a two-dimensional matrix structure to form the spectral difference feature map. The spectral difference feature map is used as a new channel and concatenated with the original channels of the visible light band image to obtain an extended channel image. The extended channel image is then subjected to convolutional dimensionality reduction operation to compress the number of channels of the extended channel image to the target number of channels, thus obtaining the spectral fusion feature.

[0024] In multispectral detection of fabric surfaces, images from different wavelengths carry complementary information about fabric defects. Visible light images can clearly reveal common visible defects such as fabric texture, color variations, and surface stains, while near-infrared images are more sensitive to latent defects such as uneven density, fiber breaks, and moisture distribution within the fabric. Effectively fusing this two types of wavelength information can significantly improve the accuracy and robustness of defect detection.

[0025] After image registration and alignment, each pixel in the visible light and near-infrared images corresponds to the same physical location on the fabric surface. For the visible light image, the light intensity value at each pixel location is extracted. When the visible light image is a color image, the pixel values ​​of the RGB channels can be converted into single-channel grayscale values ​​using a weighted average. The weighting coefficients can be the proportions in the standard grayscale conversion formula, for example, 0.299 for the red channel, 0.587 for the green channel, and 0.114 for the blue channel. The resulting grayscale value is used as the visible light response component for that pixel location. For the near-infrared image, since it is usually a single-channel image, the pixel value at the corresponding pixel location is directly extracted as the near-infrared response component.

[0026] After obtaining the two sets of response components, since the numerical ranges of the original images acquired by different sensors may differ, it is necessary to map the visible light response components and near-infrared response components to a unified numerical range. A minimum-maximum normalization strategy is employed to calculate the global minimum and maximum values ​​for the visible light and near-infrared images, respectively. For each pixel in the visible light image, its response component is subtracted from the global minimum value of that image, and then divided by the difference between the global maximum and minimum values ​​to obtain the normalized visible light response component. The same normalization operation is performed on the near-infrared image, ensuring that both types of response components fall within the numerical range of 0 to 1. This normalization process eliminates the influence of differences in sensor response curves and fluctuations in illumination intensity, making the subsequently calculated spectral difference values ​​comparable.

[0027] After normalization, the numerical difference between the visible light response component and the near-infrared response component is calculated pixel by pixel. For a pixel at position (i, j) in the image, let the normalized visible light response component be V_{i,j} and the near-infrared response component be N_{i,j}. Then, the spectral difference response value D_{i,j} at this pixel position is calculated as D_{i,j} = |V_{i,j} -N_{i,j}|. Taking the absolute value highlights the degree of difference between the two band responses, without caring which band's response is stronger. In fabric defect areas, due to abnormalities in material composition, thickness, or fiber arrangement, the response characteristics of the visible and near-infrared bands will show significant differences, resulting in a significantly higher $D_{i,j}$ value than in normal fabric areas.

[0028] The spectral difference response values ​​D_{i,j} corresponding to all pixel locations in the image are organized into a two-dimensional matrix structure to form a spectral difference feature map. The size of this feature map remains consistent with the original image, with each matrix element corresponding to the degree of spectral difference at a single pixel location. The spectral difference feature map clearly highlights areas of abnormal response in different wavelength bands on the fabric surface, which often correspond to potential defect locations. To further enhance the contrast of the feature map, it can be stretched to extend the numerical range to 0 to 255, facilitating subsequent processing and visualization analysis.

[0029] The spectral difference feature map is used as a new channel and concatenated with the existing channels of the visible light image. If the visible light image is a three-channel RGB image, the concatenation results in a four-channel image, where the first three channels retain the original RGB information, and the fourth channel is the spectral difference feature map. If the visible light image has been converted to a single-channel grayscale image, the concatenation results in a two-channel image. This channel concatenation operation preserves the original visible light texture information while introducing inter-band difference information, giving the extended channel image a stronger defect characterization capability.

[0030] Increasing the number of channels in an expanded channel image leads to a higher computational complexity in subsequent processing. To reduce the computational burden while maintaining feature representation capabilities, convolutional dimensionality reduction is performed on the expanded channel image. A single-fold convolutional kernel is used to perform pointwise convolution on the expanded channel image. This type of kernel does not change the spatial dimensions of the image, but only linearly combines the channel dimensions. Let the number of channels in the expanded channel image be C_{in}, and the number of target channels be C_{out}. Each convolutional kernel generates one output channel. The weight parameters of the convolutional kernel can be learned through pre-training or randomly initialized and then optimized end-to-end in the detection task. After the convolutional dimensionality reduction operation, the number of channels in the expanded channel image is compressed to the target number of channels, resulting in spectral fusion features.

[0031] The choice of the target number of channels needs to balance feature representation capability and computational efficiency. Typically, the target number is set between 32 and 64, which preserves key spectral difference information and texture features without resulting in an excessively large parameter scale. During convolutional dimensionality reduction, the convolutional kernels of different channels automatically learn the optimal combination between the original channels, making the compressed spectral fusion features more discriminative in defect detection tasks. To enhance the nonlinear feature extraction capability during dimensionality reduction, batch normalization layers and ReLU activation functions can be added after the convolution operation to make the distribution of spectral fusion features more stable and accelerate the convergence of subsequent detection networks.

[0032] The spectral fusion features maintain the same resolution as the original image in the spatial dimension, while integrating visible light texture information, near-infrared internal structure information, and inter-band difference information in the channel dimension. This deep fusion of multi-source information enables a clear distinction between normal fabric areas and defective areas in the feature space, providing high-quality input features for subsequent defect localization and classification. The entire spectral feature fusion process starts from the original multi-band image, and through pixel-by-pixel difference calculation, channel cascading, and convolutional dimensionality reduction, it finally generates compact and highly discriminative spectral fusion features, laying the foundation for efficient defect recognition in real-time detection scenarios.

[0033] Defect localization based on the scale-adaptive cascaded detection strategy for the spectral fusion features includes: The spectral fusion features are subjected to spatiotemporal domain difference enhancement processing. By constructing a multi-directional gradient operator, the first derivative of the spectral fusion features is calculated in the horizontal, vertical, and diagonal directions to obtain gradient response maps in four directions. The sum of squares of the magnitudes of the gradient response maps in four directions is then taken to generate a comprehensive gradient intensity feature map. The first-level detection uses a sliding window mechanism to perform a global scan on the comprehensive gradient intensity feature map. A multi-scale sliding window combination is set, including three window sizes: large, medium, and small. Each window size scans the feature map with a fixed step size. The mean and standard deviation of the gradient intensity values ​​in each window region are calculated. Candidate windows with abnormal gradient response are selected based on the joint discrimination condition of the mean and standard deviation of the gradient intensity. The spatial coordinates and size parameters of the candidate windows are used as the initial defect location box. The second-level detection refines and extracts regional features from the initial defect location box. Based on the coordinates of the initial defect location box, the corresponding region of interest feature block is cropped from the spectral fusion features. Bilateral filtering is applied to the region of interest feature block to perform edge smoothing. Foreground and background separation is performed on the smoothed feature block using an adaptive threshold segmentation algorithm. Connected components of the foreground region are extracted, and the geometric center and boundary contour of each connected component are calculated. A minimum bounding rectangle is fitted based on the boundary contour, and the coordinates of the four vertices of the minimum bounding rectangle are used as the adaptively corrected defect boundary coordinate information.

[0034] After obtaining the spectral fusion features, a multi-level detection mechanism is needed to accurately locate fabric defects. First, the spectral fusion features undergo spatiotemporal difference enhancement processing, and multi-directional gradient operators are constructed to highlight abnormal change regions in the feature map. Specifically, extended forms of the Sobel or Prewitt operators are used. In the horizontal direction, an operator template is applied to convolve the feature map; this template has differential properties in the horizontal direction, enabling the extraction of horizontal grayscale changes. In the vertical direction, a corresponding vertical operator template is applied to convolve, extracting vertical grayscale changes. Diagonally, left and right diagonal operator templates are constructed, and convolution operations are used to extract grayscale changes in the diagonal direction.

[0035] Taking the horizontal direction as an example, the operator template can be represented as follows: The template is convolved with the feature map to obtain the gradient response map in the horizontal direction. Similarly, the vertical operator template is: The vertical gradient response map is obtained. The template for the left diagonal direction operator is: The gradient response plot of the left diagonal is obtained. The template for the right diagonal direction operator is... The right diagonal gradient response plot is obtained. After obtaining the gradient response maps in four directions, the sum of squared magnitudes is calculated for each response map, i.e.: This generates a comprehensive gradient intensity feature map, which can fully reflect edge and texture variation information in all directions, providing enhanced feature representation for subsequent defect localization. In the first-level detection stage, a sliding window mechanism is used to perform a global scan of the comprehensive gradient intensity feature map. Three different window sizes are set: a large window (64×64 pixels) for detecting larger defect areas; a medium window (32×32 pixels) for detecting medium-sized defects; and a small window (16×16 pixels) for detecting small point defects. Each window size traverses the feature map with a fixed step size, which can be set to one-quarter to one-half of the window size. For example, a large window uses a step size of 16 pixels, a medium window uses 8 pixels, and a small window uses 4 pixels, ensuring complete scan coverage while considering computational efficiency. For the area covered by the current window, the gradient intensity values ​​of all pixels within that area are extracted, and the mean and standard deviation of these values ​​are calculated. The mean reflects the average gradient response level of the area, and the standard deviation reflects the dispersion of the gradient distribution. Normal fabric areas, due to their uniform texture, have relatively stable gradient responses, with both the mean and standard deviation at low levels. In contrast, defective areas, due to texture breaks or abrupt changes, exhibit significantly enhanced gradient responses, resulting in markedly higher mean and standard deviation. Therefore, a joint discrimination condition is set: when the mean gradient intensity of the window region is greater than 1.5 times the global mean and the standard deviation is greater than 1.8 times the global standard deviation, the window is judged as a candidate defect window. The coordinates of the top left corner, width, and height of all candidate windows that meet the condition are recorded to form an initial set of defect localization boxes. To avoid the same defect region being repeatedly detected by multiple overlapping windows, non-maximum suppression processing is applied to the initial localization boxes. The overlap between any two localization boxes is calculated. When the overlap exceeds a threshold, the localization box with the stronger gradient response is retained, and redundant boxes are discarded.

[0036] In the second-level detection stage, refined boundary correction is performed on the initial defect localization boxes. Based on the coordinate information of each initial localization box, the corresponding region of interest (ROI) feature blocks are cropped from the original spectral fusion feature map. Since the first-level detection uses a fixed-size sliding window, the boundaries of the localization boxes may not precisely match the actual contour of the defect, thus requiring more detailed analysis within the ROI. First, bilateral filtering is applied to the cropped feature blocks. Bilateral filtering is a non-linear filtering method that can smooth the image while preserving edge information. Bilateral filtering considers both spatial distance and pixel value differences, assigning greater weight to spatially adjacent points with similar pixel values ​​and less weight to points with large pixel value differences, thereby achieving the effect of smoothing homogeneous areas while preserving edges. The spatial standard deviation of the filter kernel can be set to 3 to 5 pixels, and the pixel value range standard deviation can be adaptively adjusted according to the dynamic range of the feature map, generally set to 10% to 15% of the feature value range. After bilateral filtering, noise within the feature blocks is effectively suppressed, and the contrast between the defect area and the background area is enhanced.

[0037] Next, foreground-background separation is performed on the smoothed feature blocks. An adaptive thresholding segmentation algorithm is used, dividing the feature block into several small local windows. The statistical characteristics of the pixel values ​​within each local window are calculated, and the segmentation threshold for that window is dynamically determined based on the local statistics. Specifically, the Otsu method or the mean-shift method can be used. The Otsu method determines the optimal threshold by maximizing the inter-class variance, maximizing the separation between the foreground and background after segmentation. The mean-shift method calculates the threshold based on the mean and standard deviation of the pixel values ​​within the local window; for example, the threshold can be set to a multiple of the local mean plus the standard deviation. Through adaptive thresholding segmentation, pixels with values ​​higher than the threshold are marked as foreground, and pixels with values ​​lower than the threshold are marked as background, resulting in a binarized segmentation result. In a binarized image, the foreground region may contain multiple independent connected components, each representing a possible defect instance. Connectivity analysis is performed on the binary image, using the eight-neighborhood connectivity criterion to group all interconnected foreground pixels into the same connected component and assign them a unique label.

[0038] For each connected component, calculate its geometric center coordinates, which can be obtained by averaging the coordinates of all pixels within the connected component. , ,in The total number of pixels within the connected component. and For the first The coordinates of each pixel are determined. Simultaneously, the boundary contours of connected components are extracted. The contour is a set of pixels at the boundary between the connected component and the background, and an ordered sequence of contour points can be obtained using a boundary tracking algorithm.

[0039] After obtaining the boundary contour of the connected component, a minimum bounding rectangle is fitted based on the coordinates of the contour points. The minimum bounding rectangle is the rectangle that completely contains the contour and has the smallest area; this rectangle may be axis-aligned or rotated. For axis-aligned rectangles, the minimum and maximum values ​​of the contour points in the horizontal and vertical directions are directly calculated, i.e. , , , The coordinates of the four vertices of the rectangle are respectively , , and For the rotated rectangle, the principal direction of the contour point set needs to be calculated using principal component analysis. The coordinate system is then rotated to the principal direction before calculating the boundary range. Finally, the rectangle coordinates are rotated back to the original coordinate system. The coordinates of the four vertices of the fitted minimum bounding rectangle are used as the corrected defect boundary coordinates. These coordinates, relative to the local coordinate system of the region of interest feature block, need to be transformed back to the global coordinate system of the original feature map. The transformation method is to add the offset of the region of interest in the global feature map to the local coordinates. After the refinement process of the second-level detection, the obtained defect boundary coordinate information more accurately matches the actual shape and position of the defect. Compared with the initial localization box of the first-level detection, the boundary position error is significantly reduced, providing a reliable spatial localization basis for subsequent defect attribute extraction and classification.

[0040] Through the above two-stage cascaded detection strategy, the first-stage detection uses a large receptive field and a sliding window mechanism to quickly identify areas that may contain defects, and the second-stage... An adaptive threshold segmentation algorithm is used to separate the foreground and background of the smoothed feature blocks. Connected components of the foreground region are extracted, and the geometric center and boundary contour of each connected component are calculated. A minimum bounding rectangle is fitted based on the boundary contour, and the coordinates of the four vertices of the minimum bounding rectangle are used as the adaptively corrected defect boundary coordinate information, including: An adaptive segmentation threshold is dynamically determined based on the global and local grayscale statistical characteristics of the smoothed feature block. The feature block is then subjected to pixel-level binarization using the adaptive segmentation threshold. Pixels with grayscale values ​​higher than the adaptive segmentation threshold are marked as foreground pixels, and pixels with grayscale values ​​lower than the adaptive segmentation threshold are marked as background pixels. Connectivity analysis is performed on the binarized feature blocks. An eight-connected neighborhood search method is used to traverse all foreground pixels, and adjacent foreground pixels are aggregated into the same connected component. The number of pixels contained in each connected component is counted and connected components are filtered. Valid connected components with a number of pixels exceeding the minimum area threshold are retained. The weighted average of the coordinates of all pixels in each valid connected component is calculated to obtain the geometric center coordinates. Boundary tracking is performed on each valid connected component to extract the outer contour point sequence. The minimum bounding rectangle is fitted to the sequence of outer contour points. The optimal rotation angle of the rectangle is determined by calculating the projection range of the contour points at different angles. The extreme values ​​of the horizontal and vertical coordinates of the contour point set are calculated at the optimal rotation angle. The coordinates of the four vertices of the minimum bounding rectangle are constructed based on the extreme values ​​of the coordinates. The coordinates of the four vertices are mapped back to the original image coordinate system through inverse rotation transformation, which serves as the adaptively corrected defect boundary coordinate information.

[0041] After obtaining the smoothed feature blocks, it is necessary to effectively separate the defect foreground from the background fabric texture. Considering the significant differences in grayscale distribution of feature blocks under different fabric types, defect types, and lighting conditions, fixed threshold segmentation methods are difficult to adapt to diverse detection scenarios. Therefore, an adaptive threshold segmentation algorithm is adopted to separate the foreground and background of the feature blocks.

[0042] The adaptive segmentation threshold is determined by comprehensively considering both the global and local gray-level statistical characteristics of the feature block. The mean gray-level value of all pixels in the feature block is calculated as the baseline parameter for the global gray-level statistical characteristics, while the standard deviation of the gray-level value of the feature block is calculated to measure the dispersion of the gray-level distribution. The feature block is divided into several sub-regions, each with a size of 16×16 pixels. The mean gray-level value and gray-level variance of each sub-region are calculated as local gray-level statistical characteristics. The mean gray-level values ​​of each sub-region are sorted, and the gray-level values ​​located at the 75th percentile are extracted as local high gray-level benchmarks. The adaptive segmentation threshold is determined by a weighted combination of the global gray-level mean and the local high gray-level benchmarks. The weighting coefficients are dynamically adjusted according to the gray-level standard deviation of the feature block. When the gray-level standard deviation is large, the weight of the local high gray-level benchmark is increased; when the gray-level standard deviation is small, the weight of the global gray-level mean is increased, ensuring that the segmentation threshold can adapt to feature blocks with different contrasts.

[0043] After obtaining the adaptive segmentation threshold, pixel-level binarization is performed on the feature block. Each pixel in the feature block is scanned line by line, and its grayscale value is read and compared with the adaptive segmentation threshold. If the pixel's grayscale value is higher than the adaptive segmentation threshold, the pixel is determined to belong to the foreground region and marked as a foreground pixel in the binarization result, with a value of 255. If the pixel's grayscale value is lower than or equal to the adaptive segmentation threshold, the pixel is determined to belong to the background region and marked as a background pixel in the binarization result, with a value of 0. After completing the binarization of all pixels, a binarized feature block containing only foreground and background pixels is obtained.

[0044] Connectivity analysis is performed on the binarized feature blocks to extract independent foreground regions. An eight-connected neighborhood search method is used to traverse all pixels in the binarized feature block. Starting from the top-left pixel of the binarized feature block, each pixel is checked sequentially from left to right and top to bottom. When an unlabeled foreground pixel with a grayscale value of 255 is encountered, the connected component growth process is initiated. Starting from this foreground pixel, pixels in its eight neighboring directions are checked, including top, bottom, left, right, and top-left, top-right, bottom-left, and bottom-right directions. If a neighboring pixel is also a foreground pixel and has not yet been labeled, it is included in the current connected component, and the eight neighbors of that neighboring pixel are checked. A depth-first search strategy is used to recursively perform neighborhood expansion until no further expansion is possible, at which point the current connected component growth is complete. A unique label number is assigned to the current connected component, and the coordinates of all pixels contained within the connected component are recorded. The remaining unlabeled pixels in the binarized feature block are traversed, and the above connected component growth process is repeated until all foreground pixels are labeled into their corresponding connected components.

[0045] After extracting connected components, the number of pixels contained in each connected component is counted. Some connected components may be generated by image noise or texture interference, with extremely small areas and no actual defect features. A minimum area threshold is set to filter connected components. The value of the minimum area threshold is determined according to the accuracy requirements of the detection task, and is usually set to 20 to 50 pixels. All connected components are traversed. If the number of pixels contained in a connected component is lower than the minimum area threshold, the connected component is determined to be invalid and discarded; if the number of pixels contained in a connected component reaches or exceeds the minimum area threshold, the connected component is determined to be valid and retained. The valid connected components after filtering are the real defect foreground regions in the feature block.

[0046] Calculate the geometric center coordinates for each valid connected component. Extract the coordinates of all pixels contained within the valid connected component. Sum the x-coordinates of all pixels and divide by the total number of pixels to obtain the x-coordinate of the geometric center; sum the y-coordinates of all pixels and divide by the total number of pixels to obtain the y-coordinate of the geometric center. The geometric center coordinates reflect the spatial location of the defect region within the feature block, providing a reference for subsequent defect localization.

[0047] Boundary tracing is performed on each valid connected component, extracting the sequence of outer contour points. An arbitrary foreground pixel is selected from the valid connected component as the starting point for boundary tracing, typically the top-left foreground pixel. Using this starting point as the current point, its eight neighboring directions are checked in a clockwise order: right, bottom-right, bottom, bottom-left, left, top-left, top, and top-right. The first neighboring pixel belonging to the current connected component is found, and this neighboring pixel is used as the next boundary point, updating the current point to this neighboring pixel. This neighboring check and boundary point update process is repeated, continuously tracing the outer boundary of the connected component until returning to the starting point, completing one complete boundary loop tracing. The coordinates of all boundary points visited sequentially during the boundary tracing process are stored in order, forming the sequence of outer contour points. The contour point sequence accurately describes the shape characteristics of the defect region.

[0048] A minimum bounding rectangle is fitted to the sequence of outer contour points to determine the smallest bounding box that can completely contain the defect area. Fitting the minimum bounding rectangle requires determining the rectangle's rotation angle, center position, and length and width dimensions. First, the projection range of the contour points at different angles is calculated to determine the optimal rotation angle. The rotation angle search range is set to 0 degrees to 90 degrees, with a step interval of 1 degree. For each candidate rotation angle, the coordinates of all points in the contour point sequence are rotated, and the extreme values ​​of the coordinates of the rotated point set in the horizontal and vertical directions are calculated. The difference between the extreme values ​​of the horizontal and vertical coordinates corresponds to the width and height of the rotated bounding rectangle, respectively, and their product is the area of ​​the bounding rectangle. All candidate rotation angles are iterated through, and the area of ​​the bounding rectangle corresponding to each angle is recorded. The rotation angle that minimizes the area of ​​the bounding rectangle is selected as the optimal rotation angle.

[0049] Under the optimal rotation angle, the contour point sequence is rotated, and the coordinates of all contour points are rotated and mapped according to the optimal rotation angle. The minimum and maximum x-coordinates of the rotated contour point set in the horizontal direction are calculated, corresponding to the x-coordinates of the left and right boundaries of the circumscribed rectangle, respectively. The minimum and maximum y-coordinates of the rotated contour point set in the vertical direction are also calculated, corresponding to the y-coordinates of the upper and lower boundaries of the circumscribed rectangle, respectively. Based on the x-coordinates of the left, right, upper, and lower boundaries, the coordinates of the four vertices of the circumscribed rectangle in the rotated coordinate system are constructed as the upper left, upper right, lower right, and lower left vertices.

[0050] Since the coordinates of the four vertices of the bounding rectangle are calculated in the rotated coordinate system, they need to be mapped back to the original image coordinate system for subsequent processing. An inverse rotation transformation is performed on each of the four vertex coordinates, with the rotation angle being the inverse of the optimal rotation angle, and the rotation center set as the geometric center of the contour point set. The four vertex coordinates obtained after the inverse rotation transformation are the vertex coordinates of the minimum bounding rectangle in the original image coordinate system. These four vertex coordinates accurately describe the spatial boundary range of the defect region and serve as the adaptively corrected defect boundary coordinate information output for subsequent defect attribute extraction and classification. Through the complete process of adaptive threshold segmentation, connected component analysis, and minimum bounding rectangle fitting described above, accurate and robust boundary localization of the defect region is achieved.

[0051] Defect attributes are extracted from the image region corresponding to the defect boundary coordinate information, and the area, aspect ratio, shape complexity, and grayscale distribution characteristics of the defect region are calculated, including: The spatial range of the defect region is determined based on the defect boundary coordinate information. In the visible light band image, the geometric center of the defect boundary coordinates is used as the seed point. The region growing algorithm is used to adaptively expand the defect region. The growth criterion is set as the gray difference between adjacent pixels is less than the dynamic growth threshold. The dynamic growth threshold is updated in real time according to the gray variance of the grown region. The complete defect connected region is obtained through the iterative growth process. The boundary of the defect connected region adaptively fits the true shape of the defect. Multi-scale morphological analysis is performed on the defect-connected region to calculate the zero-order moment and the first-order moment of the defect-connected region. The defect area is obtained through the zero-order moment, and the centroid coordinates of the defect are calculated through the first-order moment. Polar coordinate transformation is performed on the defect-connected region with the centroid as the center. The distance sequence from the defect boundary point to the centroid is calculated in the polar coordinate system. The maximum and minimum values ​​of the distance sequence are extracted as the maximum and minimum radii of the defect. The aspect ratio is obtained by calculating the ratio of the maximum and minimum radii. The second-order central moment of the defect-connected region is calculated. A covariance matrix is ​​constructed based on the second-order central moment, and its eigenvalues ​​are calculated. The anisotropy of the defect is characterized by the ratio of eigenvalues ​​as shape complexity. Multi-scale wavelet decomposition is performed on the gray values ​​in the defect-connected region to extract the energy distribution of wavelet coefficients at different scales and directions. The energy distribution vector is used as a representation of the gray-scale distribution characteristics.

[0052] After obtaining the defect boundary coordinates, refined attribute extraction of the defect region is required to support subsequent defect classification and recognition. Defect boundary coordinates are typically provided in the form of a rectangular box, including the coordinates of the top-left and bottom-right corners. However, this box often includes part of the background area, failing to accurately reflect the true spatial distribution of the defect. To obtain more accurate defect morphological features, a region growing algorithm is used to adaptively expand the defect region. Specifically, seed point coordinates are calculated based on the geometric center of the defect boundary coordinates, obtained by the arithmetic mean of the coordinates of the top-left and bottom-right corners of the rectangular box. Using the seed point as the starting position for region growing, it gradually expands outwards in the visible light band image. The region growing process uses a four-neighbor or eight-neighbor connection criterion to determine whether adjacent pixels meet the growth conditions. The growth criterion is set so that the gray-level difference between an adjacent pixel and the grown region is less than a dynamic growth threshold. This threshold is not a fixed value but is updated in real-time based on the gray-level statistical characteristics of the grown region. The initial threshold can be set as a certain percentage of the gray value of the seed point. As the growth region expands, the mean and variance of the gray values ​​of all pixels within the grown region are calculated, and the dynamic growth threshold is updated to a multiple of the current gray value variance. This adaptive threshold strategy can adapt to the non-uniformity of the gray value distribution of fabric defects, avoiding undergrowth or overgrowth caused by a fixed threshold. The iterative growth process continues until no new pixels meet the growth criteria. At this point, the obtained connected region is the complete defect region, and its boundary can adaptively fit the true shape of the defect, reflecting the spatial distribution of the defect more accurately than the original rectangular box.

[0053] After obtaining the connected region of the defect, multi-scale morphological analysis is performed on the region to extract quantitative shape features. First, the zeroth and first moments of the connected region of the defect are calculated; these moment features reflect the geometric properties of the region. The zeroth moment represents the sum of all pixels within the region, directly corresponding to the area of ​​the defect, expressed in pixels. The first moment is used to calculate the centroid coordinates of the region. The x-coordinate of the centroid is obtained by dividing the first horizontal moment by the zeroth moment, and the y-coordinate is obtained by dividing the first vertical moment by the zeroth moment. The centroid position represents the geometric center of the defect region. Unlike the rectangular frame center previously used for seed points, the centroid considers the actual shape distribution of the defect and has higher representativeness. Using the centroid as the center, a polar coordinate transformation is performed on the connected region of the defect, converting the boundary point coordinates in the Cartesian coordinate system to a polar coordinate system. For each point on the defect boundary, its radial distance and angle relative to the centroid are calculated. The radial distance is obtained by calculating the Euclidean distance between the point coordinates and the centroid coordinates, and the angle is obtained by calculating the ratio of the coordinate differences using the arctangent function. Traversing all points on the defect boundary forms a distance sequence, recording the distance variation from the defect boundary to the centroid at different angular directions. The maximum value is extracted from the distance sequence as the maximum radius of the defect, and the minimum value as the minimum radius. The maximum radius reflects the spatial scale of the defect in its most extended direction, while the minimum radius reflects its scale in its most contracted direction; the ratio of the two is the aspect ratio of the defect. The closer the aspect ratio is to 1, the closer the defect shape is to a circle or square; the further the value deviates from 1, the more pronounced the elongated or elliptical characteristics of the defect. For linear defects on the fabric surface, such as warp and weft breaks, the aspect ratio is usually large, while for point defects, such as oil stains, the aspect ratio is close to 1.

[0054] To further characterize the complexity of the defect shape, the second-order central moments of the connected regions of the defect are calculated. The second-order central moments consist of three independent components, corresponding to the lateral variance, longitudinal variance, and covariance, respectively. The lateral variance is obtained by summing the squared deviations of the x-coordinates of all pixels within the region relative to the centroid's x-coordinate; the longitudinal variance is obtained by summing the squared deviations of the y-coordinates; and the covariance is obtained by summing the products of the deviations of the x and y coordinates. Based on these three second-order central moment components, a covariance matrix is ​​constructed, which is a symmetric positive definite matrix. Eigenvalue decomposition is performed on the covariance matrix to obtain two eigenvalues. The larger eigenvalue corresponds to the principal direction of the defect, and the smaller eigenvalue corresponds to the secondary direction. The ratio of the eigenvalues ​​reflects the difference in the distribution of the defect in the two orthogonal directions, i.e., the degree of anisotropy. When the two eigenvalues ​​are nearly equal, the ratio is close to 1, indicating that the defect is relatively uniformly distributed in all directions and has a regular shape; when the two eigenvalues ​​differ significantly, the ratio is far from 1, indicating that the defect has a clear principal directionality, an irregular shape, or a complex bifurcated structure. Using the eigenvalue ratio as a quantitative indicator of shape complexity, this indicator can distinguish between defects of simple geometric shapes and defects of complex shapes. For irregular defects such as holes and fuzz on the fabric surface, the shape complexity value is relatively high.

[0055] Besides shape features, the gray-level distribution characteristics of the defect region are also an important attribute dimension. While directly using statistical measures such as the gray-level mean and standard deviation of the defect region is simple, it cannot fully reflect the spatial variation pattern of gray levels. Multi-scale wavelet decomposition is employed to perform frequency domain analysis on the gray-level values ​​within the connected regions of the defect. Using the gray-level image of the defect region as input, a two-dimensional discrete wavelet transform is applied, selecting appropriate wavelet basis functions such as the Daubechies wavelet or the Haar wavelet. Wavelet decomposition divides the image into low-frequency approximate subbands and high-frequency detail subbands. The high-frequency detail subbands are further divided into horizontal, vertical, and diagonal directions. Through multi-level decomposition, wavelet coefficients at different scale levels can be obtained. The first level of decomposition yields one low-frequency approximate subband and three high-frequency detail subbands; the second level decomposition further decomposes the low-frequency approximate subband of the first level, and so on. Typically, three to four levels of wavelet decomposition are sufficient to capture the multi-scale characteristics of gray levels. For each wavelet coefficient at each scale and in each direction, its energy is calculated; the energy is obtained by summing the squares of the coefficients. The wavelet energy across all scales and directions is organized into a vector form. The dimension of this vector depends on the number of decomposition levels and directions; for example, a three-level decomposition produces an energy vector with nine dimensions. The wavelet energy distribution vector reflects the frequency and directional characteristics of grayscale changes in the defect region. For defects with uniform texture, high-frequency energy is lower, while for defects with complex texture or sharp edges, high-frequency energy is higher. The differences in energy distribution across different directions can also reveal the anisotropic texture features of the defect, providing rich feature input for subsequent defect classification.

[0056] Through the above processing steps, the area, aspect ratio, shape complexity, and grayscale distribution characteristics of the defect region are fully extracted. Area features reflect the size of the defect, aspect ratio features characterize the directionality of the defect's extension, shape complexity features quantify the irregularity of the defect boundary, and grayscale distribution characteristics represent the texture and intensity variation patterns within the defect. These multi-dimensional defect attributes constitute a complete feature vector, providing a sufficient information foundation for subsequent matching with a pre-defined defect classification rule base, ensuring accurate identification of different types of fabric surface defects.

[0057] The defect category is determined by matching the defect attributes with a preset defect classification rule base, including: A defect classification and matching mechanism based on fuzzy membership degree is established. The defect area, aspect ratio, shape complexity, and grayscale distribution characteristics are organized into defect feature descriptors. A fuzzy membership degree function is defined for each defect category in the defect classification rule base. The fuzzy membership degree function describes the degree of belonging of the feature parameters to the category. The fuzzy membership degree value of each dimension component of the defect feature descriptor to each category is calculated. The membership degree values ​​of each dimension are fused and calculated through fuzzy inference rules. Calculate the comprehensive membership score of the defect feature descriptor for each defect category, select the defect category with the highest comprehensive membership score as the final classification result, and when the highest comprehensive membership score is lower than the confidence limit, mark the defect as a fuzzy category defect and trigger the manual review process.

[0058] After the fabric defect detection system completes defect boundary localization and extracts defect attribute parameters, a defect classification and matching mechanism based on fuzzy membership needs to be established to achieve accurate defect type identification. This mechanism fully considers the fuzziness and uncertainty of fabric defect features and can effectively handle complex situations in actual detection scenarios such as ambiguous defect boundaries and feature intersections.

[0059] First, the extracted defect area, aspect ratio, shape complexity, and grayscale distribution characteristics are organized into a unified defect feature descriptor. The defect area is normalized in pixels, with a normalization range of 0 to 1, where 1 corresponds to the maximum possible defect area within the detection field of view. The aspect ratio is calculated as the ratio of the longer side to the shorter side of the minimum bounding rectangle of the defect area. When the defect is approximately circular, the aspect ratio is close to 1; when it is strip-shaped, the aspect ratio increases significantly. Shape complexity is measured as the ratio of the square of the defect contour perimeter to the defect area. This parameter has a significant distinguishing ability for hole-like defects, fuzzy defects, and line-like defects. Grayscale distribution characteristics are characterized by calculating the statistical moments of the grayscale histogram within the defect area, including four sub-dimensions: grayscale mean, grayscale standard deviation, grayscale skewness, and grayscale kurtosis. These parameters are arranged in a fixed order to form a defect feature descriptor vector containing seven dimensions.

[0060] For each defect category defined in the defect classification rule base, a fuzzy membership function needs to be designed for each dimension of the feature descriptor. Taking holes as an example, the membership function for its area feature uses a Gaussian function, with the center position set in the medium area range, and the standard deviation parameter determined based on the typical size statistics of the hole defect. The membership function for the aspect ratio feature uses a symmetrical triangular function, with the peak position set near 1.0, reflecting that holes usually present an approximately circular or elliptical shape. The membership function for the shape complexity feature uses a one-sided trapezoidal function, maintaining a high membership value in the high complexity range, because the edges of holes usually have irregular burrs. The membership function for grayscale distribution characteristics needs to be defined separately for each of the four sub-dimensions. The membership function for the grayscale mean uses a low-value activation function, because the hole area usually exhibits a low grayscale response; the membership function for the grayscale standard deviation uses a high-value activation function, reflecting that there are large grayscale changes at the edges of the hole.

[0061] For the strip defect category, the membership function for the aspect ratio feature is designed as a high-value activation type, with the membership degree increasing rapidly when the aspect ratio is greater than 3.0. The membership function for the shape complexity feature uses a low-value activation type because strip defects typically exhibit a regular linear shape. Based on the characteristic that strip defects are usually caused by missing warp or weft yarns, the peak value of the grayscale distribution feature's membership function is set in a relatively low range.

[0062] For stain-type defects, the membership function for area features covers a wider range of values ​​because stain sizes vary considerably. The membership function for shape complexity is set to activate within a medium value range, resulting in relatively smooth stain boundaries, though not as regular as those for geometrically regular defects. In the grayscale distribution features, the membership function for grayscale standard deviation uses a low-value activation type, reflecting the relatively uniform grayscale within the stain area.

[0063] After defining the membership functions for each dimension, the membership values ​​of the input defect feature descriptors to each defect category are calculated dimension by dimension. For the first dimension of the feature descriptor, the area parameter, this value is substituted into the area membership function corresponding to each defect category to obtain the membership values ​​of the area to categories such as holes, stripes, and stains. This process is repeated to complete the membership calculation for all seven dimensions, ultimately resulting in a membership matrix with a dimension of 7 multiplied by the total number of categories.

[0064] The design of fuzzy inference rules needs to consider the different importance of each dimension of features to defect classification. For hole-type defects, shape complexity and grayscale standard deviation have relatively high discriminative weights; for strip-shaped defects, aspect ratio has a dominant discriminative weight; and for stain-type defects, grayscale distribution characteristics have a more critical discriminative weight. Based on domain expert experience, feature dimension weight coefficient vectors are established. The weight coefficient vector for hole-type defects is: area 0.15, aspect ratio 0.10, shape complexity 0.30, grayscale mean 0.15, grayscale standard deviation 0.25, grayscale skewness 0.03, and grayscale kurtosis 0.02. The weight coefficient vector for strip-shaped defects is: area 0.10, aspect ratio 0.45, shape complexity 0.20, grayscale mean 0.10, grayscale standard deviation 0.08, grayscale skewness 0.04, and grayscale kurtosis 0.03. The weight coefficient vector for the stain class has the following parameters: area 0.20, aspect ratio 0.08, shape complexity 0.12, grayscale mean 0.25, grayscale standard deviation 0.20, grayscale skewness 0.08, and grayscale kurtosis 0.07.

[0065] The membership scores of each dimension are weighted and summed with their corresponding weight coefficients to obtain the comprehensive membership score of the defect feature descriptor for each defect category. A fuzzy operator is introduced during the calculation to perform a nonlinear transformation on the membership scores, enhancing the contribution of high-membership features and suppressing the interference of low-membership features. Specifically, an exponential enhancement operator is used to perform a power transformation on the membership scores of each dimension, with the exponent set to 1.5. This results in feature dimensions with membership scores above 0.8 receiving greater weight, while the influence of feature dimensions with membership scores below 0.3 is significantly weakened.

[0066] After calculating the comprehensive membership scores for all categories, the score vectors are sorted, and the category with the highest score is selected as the preliminary classification result. To ensure classification reliability, a confidence threshold of 0.60 is set. When the highest comprehensive membership score exceeds this threshold, the defect category is confirmed as the final classification result, and a category label is output. If the highest comprehensive membership score is lower than the confidence threshold, it indicates that the defect feature does not match significantly with any category in the rule base, and may belong to a feature crossover defect or a novel defect pattern. In this case, the defect is marked as a fuzzy category defect, labeled with a special identifier in the detection result report, and a manual review process is triggered.

[0067] The manual review process involves pushing image data, location coordinates, feature parameters, and membership scores for each category of fuzzy-category defects to the quality control personnel's workstations, where experienced quality control personnel manually assess the defects. The category labels and feature data confirmed by the quality control personnel are then fed back to the defect classification rule base to optimize the fuzzy membership function parameters and weight coefficients, enabling continuous improvement of the classification model. For samples identified as novel defect patterns, the system automatically creates new defect category entries, initializes the membership function and weight coefficients for that category, and enables the new category in subsequent detections, thus giving the classification system adaptive expansion capabilities.

[0068] The aforementioned defect classification and matching mechanism based on fuzzy membership can effectively handle the fuzziness and category overlap of fabric defect features. While ensuring classification accuracy, it enables reliable screening of uncertain samples through a confidence discrimination mechanism. Combined with a manual review process, it forms a human-machine collaborative quality control closed loop, significantly improving the practicality and reliability of the fabric defect detection system.

[0069] A second aspect of the present invention provides a real-time fabric defect detection system integrating machine vision and deep learning, comprising: An image acquisition unit is used to acquire multispectral image data of the fabric surface. The multispectral image data includes visible light band images and near-infrared band images. The visible light band images and near-infrared band images are registered and aligned to obtain an aligned multispectral image. The spectral feature unit is used to perform spectral feature fusion on the multi-band image. It constructs a spectral difference feature map by calculating the spectral response difference between the visible light band image and the near-infrared band image pixel by pixel. The spectral difference feature map is channel-cascaded with the visible light band image, and the cascaded multi-channel data is mapped to spectral fusion features by dimension reduction projection. The defect localization unit is used to locate defects in the spectral fusion features based on a scale-adaptive cascaded detection strategy. The first-level detection uses a large receptive field detection network to perform a global scan of the spectral fusion features, identify regions with abnormal feature responses, and output an initial defect localization box. The second-level detection dynamically adjusts the receptive field range according to the size of the initial defect localization box to obtain adaptively corrected defect boundary coordinate information. The defect attribute unit is used to extract defect attributes from the image region corresponding to the defect boundary coordinate information, calculate the area, aspect ratio, shape complexity and grayscale distribution characteristics of the defect region, and match the defect attributes with a preset defect classification rule library to determine the defect category.

[0070] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0071] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0072] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time fabric defect detection method integrating machine vision and deep learning, characterized in that, include: Multispectral image data of the fabric surface is acquired, including visible light band images and near-infrared band images. The visible light band images and near-infrared band images are registered and aligned to obtain an aligned multispectral image. The multi-band images are subjected to spectral feature fusion. The difference in spectral response between the visible light band image and the near-infrared band image is calculated pixel by pixel to construct a spectral difference feature map. The spectral difference feature map is then concatenated with the visible light band image, and the concatenated multi-channel data is mapped to spectral fusion features through dimensionality reduction projection. Defect localization is performed on the spectral fusion features based on a scale-adaptive cascaded detection strategy. The first-level detection uses a large receptive field detection network to perform a global scan of the spectral fusion features, identify regions with abnormal feature responses, and output an initial defect localization box. The second-level detection dynamically adjusts the receptive field range according to the size of the initial defect localization box to obtain adaptively corrected defect boundary coordinate information. Defect attributes are extracted from the image region corresponding to the defect boundary coordinate information. The area, aspect ratio, shape complexity, and grayscale distribution characteristics of the defect region are calculated. The defect attributes are matched with a preset defect classification rule library to determine the defect category.

2. The method according to claim 1, characterized in that, The multi-band images are subjected to spectral feature fusion. A spectral difference feature map is constructed by calculating the spectral response difference between the visible light band image and the near-infrared band image pixel-by-pixel. The spectral difference feature map is then concatenated with the visible light band image, and the concatenated multi-channel data is mapped to spectral fusion features through dimensionality reduction projection. This includes: The light intensity values ​​of each pixel in the visible light band image are extracted as visible light response components, and the light intensity values ​​of the corresponding pixel in the near-infrared band image are extracted as near-infrared response components. The visible light response components and the near-infrared response components are mapped to a unified numerical range, and the numerical difference between the visible light response components and the near-infrared response components is calculated. The numerical difference is used as the spectral difference response value of the corresponding pixel location. The spectral difference response values ​​of all pixel locations are organized into a two-dimensional matrix structure to form the spectral difference feature map. The spectral difference feature map is used as a new channel and concatenated with the original channels of the visible light band image to obtain an extended channel image. The extended channel image is then subjected to convolutional dimensionality reduction operation to compress the number of channels of the extended channel image to the target number of channels, thus obtaining the spectral fusion feature.

3. The method according to claim 1, characterized in that, Defect localization based on the scale-adaptive cascaded detection strategy for the spectral fusion features includes: The spectral fusion features are subjected to spatiotemporal domain difference enhancement processing. By constructing a multi-directional gradient operator, the first derivative of the spectral fusion features is calculated in the horizontal, vertical, and diagonal directions to obtain gradient response maps in four directions. The sum of squares of the magnitudes of the gradient response maps in four directions is then taken to generate a comprehensive gradient intensity feature map. The first-level detection uses a sliding window mechanism to perform a global scan on the comprehensive gradient intensity feature map. A multi-scale sliding window combination is set, including three window sizes: large, medium, and small. Each window size scans the feature map with a fixed step size. The mean and standard deviation of the gradient intensity values ​​in each window region are calculated. Candidate windows with abnormal gradient response are selected based on the joint discrimination condition of the mean and standard deviation of the gradient intensity. The spatial coordinates and size parameters of the candidate windows are used as the initial defect location box. The second-level detection refines and extracts regional features from the initial defect location box. Based on the coordinates of the initial defect location box, the corresponding region of interest feature block is cropped from the spectral fusion features. Bilateral filtering is applied to the region of interest feature block to perform edge smoothing. Foreground and background separation is performed on the smoothed feature block using an adaptive threshold segmentation algorithm. Connected components of the foreground region are extracted, and the geometric center and boundary contour of each connected component are calculated. A minimum bounding rectangle is fitted based on the boundary contour, and the coordinates of the four vertices of the minimum bounding rectangle are used as the adaptively corrected defect boundary coordinate information.

4. The method according to claim 3, characterized in that, An adaptive threshold segmentation algorithm is used to separate the foreground and background of the smoothed feature blocks. Connected components of the foreground region are extracted, and the geometric center and boundary contour of each connected component are calculated. A minimum bounding rectangle is fitted based on the boundary contour, and the coordinates of the four vertices of the minimum bounding rectangle are used as the adaptively corrected defect boundary coordinate information, including: An adaptive segmentation threshold is dynamically determined based on the global and local grayscale statistical characteristics of the smoothed feature block. The feature block is then subjected to pixel-level binarization using the adaptive segmentation threshold. Pixels with grayscale values ​​higher than the adaptive segmentation threshold are marked as foreground pixels, and pixels with grayscale values ​​lower than the adaptive segmentation threshold are marked as background pixels. Connectivity analysis is performed on the binarized feature blocks. An eight-connected neighborhood search method is used to traverse all foreground pixels, and adjacent foreground pixels are aggregated into the same connected component. The number of pixels contained in each connected component is counted and connected components are filtered. Valid connected components with a number of pixels exceeding the minimum area threshold are retained. The weighted average of the coordinates of all pixels in each valid connected component is calculated to obtain the geometric center coordinates. Boundary tracking is performed on each valid connected component to extract the outer contour point sequence. The minimum bounding rectangle is fitted to the sequence of outer contour points. The optimal rotation angle of the rectangle is determined by calculating the projection range of the contour points at different angles. The extreme values ​​of the horizontal and vertical coordinates of the contour point set are calculated at the optimal rotation angle. The coordinates of the four vertices of the minimum bounding rectangle are constructed based on the extreme values ​​of the coordinates. The coordinates of the four vertices are mapped back to the original image coordinate system through inverse rotation transformation, which serves as the adaptively corrected defect boundary coordinate information.

5. The method according to claim 1, characterized in that, Defect attributes are extracted from the image region corresponding to the defect boundary coordinate information, and the area, aspect ratio, shape complexity, and grayscale distribution characteristics of the defect region are calculated, including: The spatial range of the defect region is determined based on the defect boundary coordinate information. In the visible light band image, the geometric center of the defect boundary coordinates is used as the seed point. The region growing algorithm is used to adaptively expand the defect region. The growth criterion is set as the gray difference between adjacent pixels is less than the dynamic growth threshold. The dynamic growth threshold is updated in real time according to the gray variance of the grown region. The complete defect connected region is obtained through the iterative growth process. The boundary of the defect connected region adaptively fits the true shape of the defect. Multi-scale morphological analysis is performed on the defect-connected region to calculate the zero-order moment and the first-order moment of the defect-connected region. The defect area is obtained through the zero-order moment, and the centroid coordinates of the defect are calculated through the first-order moment. Polar coordinate transformation is performed on the defect-connected region with the centroid as the center. The distance sequence from the defect boundary point to the centroid is calculated in the polar coordinate system. The maximum and minimum values ​​of the distance sequence are extracted as the maximum and minimum radii of the defect. The aspect ratio is obtained by calculating the ratio of the maximum and minimum radii. The second-order central moment of the defect-connected region is calculated. A covariance matrix is ​​constructed based on the second-order central moment, and its eigenvalues ​​are calculated. The anisotropy of the defect is characterized by the ratio of eigenvalues ​​as shape complexity. Multi-scale wavelet decomposition is performed on the gray values ​​in the defect-connected region to extract the energy distribution of wavelet coefficients at different scales and directions. The energy distribution vector is used as a representation of the gray-scale distribution characteristics.

6. The method according to claim 5, characterized in that, The defect category is determined by matching the defect attributes with a preset defect classification rule base, including: A defect classification and matching mechanism based on fuzzy membership degree is established. The defect area, aspect ratio, shape complexity, and grayscale distribution characteristics are organized into defect feature descriptors. A fuzzy membership degree function is defined for each defect category in the defect classification rule base. The fuzzy membership degree function describes the degree of belonging of the feature parameters to the category. The fuzzy membership degree value of each dimension component of the defect feature descriptor to each category is calculated. The membership degree values ​​of each dimension are fused and calculated through fuzzy inference rules. Calculate the comprehensive membership score of the defect feature descriptor for each defect category, select the defect category with the highest comprehensive membership score as the final classification result, and when the highest comprehensive membership score is lower than the confidence limit, mark the defect as a fuzzy category defect and trigger the manual review process.

7. A real-time fabric defect detection system integrating machine vision and deep learning, used to implement the method as described in any one of claims 1-6, characterized in that, include: An image acquisition unit is used to acquire multispectral image data of the fabric surface. The multispectral image data includes visible light band images and near-infrared band images. The visible light band images and near-infrared band images are registered and aligned to obtain an aligned multispectral image. The spectral feature unit is used to perform spectral feature fusion on the multi-band image. It constructs a spectral difference feature map by calculating the spectral response difference between the visible light band image and the near-infrared band image pixel by pixel. The spectral difference feature map is channel-cascaded with the visible light band image, and the cascaded multi-channel data is mapped to spectral fusion features by dimension reduction projection. The defect localization unit is used to locate defects in the spectral fusion features based on a scale-adaptive cascaded detection strategy. The first-level detection uses a large receptive field detection network to perform a global scan of the spectral fusion features, identify regions with abnormal feature responses, and output an initial defect localization box. The second-level detection dynamically adjusts the receptive field range according to the size of the initial defect localization box to obtain adaptively corrected defect boundary coordinate information. The defect attribute unit is used to extract defect attributes from the image region corresponding to the defect boundary coordinate information, calculate the area, aspect ratio, shape complexity and grayscale distribution characteristics of the defect region, and match the defect attributes with a preset defect classification rule library to determine the defect category.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images