Textile surface defect detection method and system based on image features

Abstract

The present application provides a textile surface defect detection method and system based on image features, relates to the technical field of textile surface defect detection, and acquires an RGB image of a textile to be detected, and outputs at least five semantic labels after being scaled to 512*512 pixels through a semantic segmentation model. Hyperspectral imaging with a wavelength of 400-1700 nm and a resolution of 10 nm is adopted, pixel reflectivity is calculated through a reflectivity formula, reflectivity of 5-8 characteristic bands is compared, and if the average deviation exceeds a threshold value, it is marked as a contaminant. The type and location of the stain are matched by spectral angle distance, standard samples are converted to LAB color space after being converted to the same semantic region, a rotating minimum bounding rectangle is extracted and a grid is divided, invalid pixels are filled with a weighted average of 3*3 neighboring valid pixels, and the average LAB value of the grid is calculated. The contaminant pixels of the grid corresponding to the image to be detected are filled with the average value of the standard sample, and then the A and B channels are clustered through a sliding window, the pilling area is preliminarily judged according to the standard deviation of the L channel, and the pilling area is confirmed by comparing with a threshold value in combination with the three-dimensional point cloud of the structured light.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of textile surface defect detection, in particular to a textile surface defect detection method and system based on image features. BACKGROUND

[0002] With the rapid development of large-scale production of the clothing industry and e-commerce retail, traditional manual detection for textile defect detection relies on experience judgment, which has strong subjectivity, low efficiency, high omission rate and other problems, and has been difficult to meet the needs of industrialized batch detection. In recent years, the application of computer vision has realized the automatic identification of textile quality. Through machine learning model analysis of clothing images, the positioning and classification of defects can be completed without human intervention, which significantly improves the detection efficiency and reduces the labor cost.

[0003] The existing technology for textile defect detection is mostly based on machine vision and deep learning. Through industrial cameras to collect clothing RGB images, deep learning models are used to extract image features to realize defect recognition and positioning. For example, through the color difference of the RGB image to identify obvious stains, or through the texture feature change to judge the pilling area. Some solutions will combine industrial-grade cameras to improve image resolution, or through multi-device linkage such as light source control system and conveyor belt optimization imaging environment to reduce external interference. However, the popularization and application of these technologies are still limited by multiple factors. On the one hand, the sensitivity to subtle defects is insufficient. RGB images can only capture two-dimensional color and texture information. For early-stage micro-pilling, subtle defects such as stains with a diameter of less than 1mm and light stains are easily hidden by background texture due to unobvious features, resulting in low detection rate. On the other hand, the method of global modeling not only requires expensive industrial high-resolution cameras, special light source systems and other equipment, but also has low efficiency, which is not suitable for e-commerce return detection, small and medium batch production quality inspection and other scenes.

[0004] Therefore, the industry urgently needs a method that can reduce the dependence on high-end hardware, adapt to diversified detection scenarios, and accurately model defects in different areas of clothing to improve the detection rate of subtle defects.

[0005] The above information disclosed in the background section is only used to enhance the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The purpose of the present application is to provide a textile surface defect detection method and system based on image features to solve the problems raised in the background.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] The method for detecting defects on a textile surface based on image features includes the following specific steps:

[0009] Step 1: Obtain the RGB images of the textile to be detected and the standard sample of the textile, uniformly scale the images to a resolution of 512*512 pixels, and then use the trained recognition model to perform semantic segmentation, outputting the semantic labels corresponding to each pixel.

[0010] Step 2: Obtain the multispectral reflectance curve of the standard material in the range of 400-1700 nm, compare it with the multispectral reflectance curve of each pixel of the textile to be detected, determine the reflectance deviation of each characteristic wavelength, and mark the pixels as contaminated when the reflectance deviation of multiple characteristic wavelengths exceeds the deviation threshold. The remaining pixels are marked as non-contaminated. Meanwhile, match the spectrum of the contaminated pixels with the known stain spectrum to output the stain type and pixel position.

[0011] Step 3: Convert the RGB images generated by the semantic labels of the standard sample of the textile and the textile to be detected into the LAB color space, extract the rotating minimum bounding rectangle, evenly divide the grid matrix and sequentially number them, fill the invalid pixels in the grid with the weighted average of the valid pixels in their neighborhood, calculate the mean value of the LAB channel in each grid, and fill the contaminated pixels with the mean value of the LAB channel of the corresponding grid of the standard sample of the textile according to their grid number position.

[0012] Step 4: Slide a window of a predetermined size in the LAB color space with the semantic labels, extract the color features of the pixels within the window to complete fast clustering, and output the standard deviation of each L channel under different clustering results. Compare the preliminary judged pilling area with the preset threshold.

[0013] Step 5: For the preliminary judged pilling area, use structured light imaging technology to obtain three-dimensional point cloud data, and calculate the curvature features in the area based on the three-dimensional point cloud data. When the curvature features match the spherical features, it is determined as a pilling area.

[0014] Further, determining the reflectance deviation of each characteristic wavelength and marking the pixels as contaminated when the reflectance deviation of multiple characteristic wavelengths exceeds the deviation threshold includes the following steps:

[0015] Using hyperspectral imaging technology, the spectral range is set to 400-1700 nm, and the resolution is set to 10 nm. A hyperspectral image data cube is constructed, and the reflectance light signal intensity of each pixel on the surface of the textile to be detected at each wavelength is directly obtained through a light splitting component and a detector. The average value of the reflectance light signal intensity of all pixels is calculated. The calculation method of reflectance is as follows: the ratio of the reflectance light signal intensity of the textile to be detected at a certain wavelength to the reflectance light signal intensity of a standard white plate under the same lighting conditions and wavelength.

[0016] Obtain the reflectivity curve of each standard material in the 400-1700nm wave band from the material spectral reference library, and extract the reflectivity of 5-8 characteristic wave bands of the reference sample based on the unique spectral characteristics of each standard material, forming a standard material-characteristic wave band-reflectivity control table;

[0017] Compare the characteristic wave band reflectivity of each pixel with the corresponding characteristic wave band of all standard materials in the control table, and calculate the matching degree, wherein the calculation method of the matching degree is: the absolute value of the difference between the reflectivity of the characteristic wave band of a certain pixel of the textile to be detected and the reflectivity of the characteristic wave band of a certain standard material, and then calculate the average deviation based on the matching degree, that is, the average value of the matching degrees of all characteristic wave bands, and set a deviation threshold value, when the average deviation calculated under each standard material is greater than the deviation threshold value, the pixel is determined as a contaminant pixel.

[0018] Further, the method for outputting the stain type and pixel position by matching the spectrum of the contaminant pixel with the known stain spectrum is:

[0019] Obtain the published stain spectrum database, and match the full-wave band reflectivity curve of each contaminant pixel with the standard reflectivity curve of the stain spectrum database wave by wave, and calculate the spectral angle distance of the two, when the spectral angle distance of a certain pixel and a certain type of stain in the database is less than a preset threshold, output the type of the stain and the position of the pixel.

[0020] Further, the method for converting the textile standard sample and the RGB image generated by the semantic label into LAB color space is:

[0021] For the RGB image of the textile standard sample and the textile to be detected, based on the pre-labeled semantic label, the image sub-blocks of each semantic region are intercepted by the image segmentation algorithm, so that each image sub-block only contains pixel information of a single semantic region, and the extracted image sub-blocks of each semantic region are converted pixel by pixel using the color space conversion formula, and the LAB color data is output.

[0022] Further, the method for extracting a rotating minimum bounding rectangle and then uniformly dividing the grid matrix and sequentially numbering is:

[0023] For the image sub-blocks of each semantic region intercepted based on the pre-labeled semantic label by the image segmentation algorithm, for image sub-blocks of the same semantic region, define the pixel coordinate set of the image sub-blocks as Wherein, n1 represents the total number of pixels under this semantic label;

[0024] Take the average coordinates of all pixels as the region center O(x c , y c), all pixel coordinates are centralized, the centralized horizontal coordinate is the original horizontal coordinate minus the horizontal coordinate of the region center, and the centralized vertical coordinate is the original vertical coordinate minus the vertical coordinate of the region center;

[0025] The principal axis slope is obtained by the least square method, the slope is calculated by the sum of the product of all centralized horizontal coordinates and vertical coordinates divided by the sum of squares of all centralized horizontal coordinates, the rotation angle is the inverse tangent value of the slope, the distance between the two pixels on the principal axis is taken as the length of the rotated rectangle, the distance between the two pixels in the direction perpendicular to the principal axis is taken as the width of the rotated rectangle, the center O is taken as the origin, the rotated rectangle length is offset by half in the principal axis direction, and the rotated rectangle width is offset by half in the direction perpendicular to the principal axis, to form the four vertices of the rotated minimum bounding rectangle;

[0026] The local coordinate system is constructed with the center of the rotated minimum bounding rectangle as the origin, the principal axis direction as the u-axis, and the direction perpendicular to the principal axis as the v-axis, the rotated minimum bounding rectangle is uniformly divided into a grid matrix with dimensions a*b, the image sub-blocks of the same semantic region have consistent grid numbers, and a, b>0, the boundary of the rectangle in the coordinate system is [u min , u max ]x[v min , v max ], wherein the size of the grid in the u-axis direction is:

[0027]

[0028] In the formula, s u represents the size of the grid in the u-axis direction;

[0029] The size of the grid in the v-axis direction is:

[0030]

[0031] In the formula, s v represents the size of the grid in the v-axis direction, the divided grid matrix is sequentially numbered, the numbering rule is that the first grid in the upper left corner of the rotated minimum bounding rectangle is numbered from left to right in the u-axis direction, when the numbering of a row of grids is completed, the leftmost grid in the next row is numbered from left to right, and all grids are assigned unique numbers.

[0032] Further, the invalid pixels in the grid are filled by the weighted average of the valid pixels in the neighborhood as follows:

[0033] The pixels beyond the original semantic region are regarded as invalid pixels. A 3*3 pixel square region is constructed with the invalid pixel P0(x0, y0) as the origin. The pixels belonging to the current semantic region in the square region are screened and recorded as the valid pixel set E. When the 3*3 pixel square region has no valid pixel, the side length of the square neighborhood is increased by 2 pixels, i.e. the square neighborhood is expanded by 1 pixel upwards, downwards, leftwards and rightwards. This process is repeated until at least one valid pixel is found. The pixel distance between the valid pixel P j (x j , y j ) in the final neighborhood and the invalid pixel is calculated.

[0034] d j = |x j -x0||y j -y0|

[0035] In the formula, d j represents the pixel distance between the valid pixel and the invalid pixel, j represents the index of the valid pixel, j = 1, 2, …, n2, and n2 represents the number of valid pixels.

[0036] The reciprocal of the pixel distance between the valid pixel and the invalid pixel is accumulated to obtain d total , and the weight of each valid pixel is calculated.

[0037]

[0038] In the formula, ω j represents the weight of each valid pixel.

[0039] For the invalid pixel, the LAB channels are respectively executed to perform weighted calculation and filling.

[0040]

[0041] In the formula, L0, A0 and B0 represent the filling values of the invalid pixel, and L j , A j and B j represent the LAB three-channel values of the valid pixel.

[0042] Further, the window of the preset size is slid in the LAB color space of the same semantic label, and the color features of the pixels in the window are extracted to complete the method for fast clustering:

[0043] A sliding window of w*w pixels is set, and the sliding traversal is performed in the image sub-block of the same semantic region converted into the LAB color space. The step is set as For the case that there are not enough pixels for one window size for the edge area, for horizontal sliding, when the current column start position plus the window width exceeds the right boundary of the semantic area, the right boundary of the window is fixed at the last column, and the position is taken as the termination position of the current window, and no longer continue to move right; for vertical sliding, when the current row start position plus the window height exceeds the lower boundary of the semantic area, the bottom of the window is fixed at the last row, and the position is taken as the termination position of the current window, and no longer continue to move down; for the area covered by each window, the LAB three-channel feature values of all pixels in the window are extracted, only the A and B channels are reserved as clustering features, and the Silhouette Score profile coefficient algorithm is used to output the clustering result value K.

[0044] Further, the standard deviation of each L channel is output under different clustering results, and the method for outputting the preliminary judged pilling area by comparing the preset threshold value is:

[0045] In the sliding window, for each clustering result, the L channel values of all pixels in each clustering cluster are extracted, and the standard deviation of the L channel values of each clustering cluster is calculated. A threshold value is set in advance. When the standard deviation is greater than the threshold value, it is considered that the pixels of the sliding window are the preliminary judged pilling area.

[0046] Further, the method for determining the pilling area when the curvature feature in the region is consistent with the spherical feature is:

[0047] For the actual position of the preliminary judged pilling area, the three-dimensional point cloud data of the region is obtained by the structured light imaging technology:

[0048]

[0049] In the formula, Q represents the entire local point cloud data set, q m′ represents the m'th point of the point cloud data, (x m′ , y m′ , z m′ ) represents the coordinates of the m'th point of the point cloud data, N0 represents the total number of points in the point cloud data, m' represents the point cloud index, m' = 1, 2, …, N0;

[0050] For each point q m′ , select the nearest 20 points in its space to form a neighborhood point set

[0051] For the neighborhood point set , perform PCA to calculate the covariance matrix:

[0052]

[0053] In the formula, C m′Representing point q m′ The local covariance matrix, q represents the center point of the neighborhood point set. n′ Let n' represent the center point of the neighborhood, where n' represents the index of the set of neighborhood points, n' = 1, 2, ..., 20.

[0054] For the covariance matrix C m′ Eigenvalue decomposition yields three eigenvalues ​​in ascending order: λ0, λ1, λ2, where λ0 ≤ λ1 ≤ λ2. Based on these eigenvalues, a formula for calculating curvature is constructed.

[0055]

[0056] In the formula, ql represents curvature;

[0057] Then, for each point q m′ Only its neighborhood point set is used Perform spherical fitting:

[0058] (xa′) 2 +(yb′) 2 +(zc′) 2 =r 2

[0059] In the formula, (a′, b′, c′) represent the three-dimensional coordinates of the center of the sphere, and r represents the radius of the sphere.

[0060] The solution is found by minimizing the objective function of the error, where the formula for minimizing the objective function is:

[0061]

[0062] The optimal parameters (a′, b′, c′, r) are solved using the nonlinear least squares method. For each point, the sphericity is calculated.

[0063]

[0064] In the formula, S represents sphericity;

[0065] Calculate the fitting error:

[0066]

[0067] In the formula, E represents the fitting error, x n′ y n′ , z n′ Represents the neighborhood point set The three-dimensional coordinates of the n′-th point in the middle;

[0068] For each point q m′When ql>0.8, S>0.7, E<0.01, 0.25<=r<=1.5 exist, the pilling region of the preliminary judgment is considered to belong to the pilling region.

[0069] The application further provides an image feature-based textile surface defect detection system for executing the image feature-based textile surface defect detection method.

[0070] The semantic segmentation module is configured to acquire RGB images of the textile to be detected and a textile standard sample, uniformly scale the images to a resolution of 512*512 pixels, and perform semantic segmentation on the images using the trained recognition model to output semantic labels corresponding to each pixel.

[0071] The spectral detection module is configured to acquire a multispectral reflection curve of the standard material in a range of 400-1700 nm, compare the multispectral reflection curve with the multispectral reflection curve of each pixel of the textile to be detected, determine a reflection deviation of each characteristic waveband, mark a pixel as a contaminant pixel when the reflection deviation of the multiple characteristic wavebands exceeds a deviation threshold, mark the remaining pixels as non-contaminant pixels, and simultaneously match the spectrum of the contaminant pixel with a known stain spectrum to output a stain type and a pixel position.

[0072] The contaminated pixel filling module is configured to convert the textile standard sample and the RGB image generated based on the semantic labels into an LAB color space, extract a rotated minimum bounding rectangle, uniformly divide a grid matrix and sequentially number the grid matrix, fill invalid pixels in the grid matrix with a weighted average of valid pixels in the neighborhood of the invalid pixels, calculate an average value of the LAB channels in each grid, and fill the contaminant pixels with the average value of the LAB channels of the corresponding grid of the textile standard sample according to the grid number position of the contaminant pixels.

[0073] The pilling preliminary judgment module is configured to slide a window of a preset size in the LAB color space based on the semantic labels, extract color features of pixels in the window to complete fast clustering, output a standard deviation of each L channel under different clustering results, and compare the standard deviation with a preset threshold to output a preliminary pilling region.

[0074] The pilling confirmation module is configured to acquire three-dimensional point cloud data using a structured light imaging technology for the preliminary pilling region, calculate a curvature feature in the region based on the three-dimensional point cloud data, and determine the region as a pilling region when the curvature feature is consistent with a spherical feature.

[0075] Compared with the prior art, the application has the following advantages:

[0076] The present application is based on the division of different regions of the textile by semantic labels, the detection and positioning of stains are determined by hyperspectral imaging technology, and then the pixels determined as stains are filled according to the standard sample of the textile, so as to avoid the interference of the stain area on the subsequent brightness channel analysis, ensure that the brightness fluctuation characteristics of the pilling area can be accurately captured, and after the image of the textile to be detected is converted into LAB color space, a window of a preset size is slid, the standard deviation of each L channel under different clustering results is output based on the color characteristics, compared with the threshold value, and the suspected pilling part is preliminarily positioned, and then for the preliminarily positioned area, whether it is a pilling area is judged through three-dimensional point cloud data, the whole textile is not modeled in three dimensions, and the accuracy of pilling detection is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 It is a whole method flowchart of the present application;

[0078] Figure 2 It is an L channel standard deviation result graph of the clustering cluster of the present application;

[0079] Figure 3 It is a curvature and radius index graph of the present application;

[0080] Figure 4 It is a sphericity and fitting error index graph of the present application;

[0081] Figure 5 It is a whole system structure schematic diagram of the present application. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical scheme and advantages of the present application more clear and obvious, the present application is further described in detail below in combination with specific embodiments.

[0083] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present application should be understood as the usual meaning understood by those skilled in the art to which the present application belongs. The "first", "second" and similar words used in the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connected" or "connected" and similar words are not limited to physical or mechanical connection, but can include electrical connection, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent the relative positional relationship, when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0084] EMBODIMENT:

[0085] Please refer toFigures 1 to 4 The application provides a technical solution:

[0086] The image feature-based textile surface defect detection method comprises the following specific steps:

[0087] Step 1: Obtain the RGB images of the textile to be detected and the standard sample of the textile, uniformly scale the images to 512*512 pixel resolution, and then use the trained recognition model to perform semantic segmentation, and output the semantic labels corresponding to each pixel point;

[0088] DeepFashion2 Dataset is used as training data. The dataset contains a large number of textile images labeled with clothing categories, components and attributes, including 13 categories of clothing such as shirts, pants, dresses, coats and skirts. On this basis, the components of the clothes are further subdivided, such as collars, cuffs, pockets, waistbands and skirts. Each pixel is only labeled with one label. At least 1000 image samples covering multiple categories of textiles such as shirts, pants and socks are selected from the dataset. The pixel mask belongs to multi-class pixel-level labeling, for example, collar pixels, cuff pixels, etc. Without additional manual labeling, at least 5 preset semantic labels are output for each pixel, for example, for jeans, the pixels are labeled with waistband, leg, pocket, crotch and leg; for shirts, the pixels are labeled with body, collar, cuff, pocket, hem, shoulder, buttonhole, waistband;

[0089] Preprocess all images: retain the main body of the textile, uniformly crop to a square region in the zero-padded manner, and then scale to 512*512 pixel resolution;

[0090] Training process: the training batch size is set to 16, the cross-entropy loss function is used as the optimization objective, and the Adam optimizer is used, wherein the initial learning rate of the Adam optimizer is set to 0.001, the training is iterated on the training set of DeepFashion2 Dataset for 100 epochs, the learning rate is reduced to 1 / 10 of the original every 10 epochs, when the average intersection over union (mIoU) of the validation set provided by DeepFashion2 Dataset is higher than 0.85 for 5 consecutive epochs, it is considered to be stable, and the training is terminated in advance, if the condition is not met within 100 epochs, the maximum epoch is terminated;

[0091] The U-Ne semantic segmentation model is constructed: in the encoder part, 4 layers of convolution operation are adopted, each layer contains 2 times of 3*3 convolution, followed by ReLU activation after each convolution, and finally followed by 1 time of 2*2 maximum pooling, and the output channel number of the 4 layers of convolution is 64, 128, 256 and 512 in turn; in the decoder part, 4 layers of deconvolution operation are adopted, each layer contains a 3*3 transposed convolution kernel and a ReLU activation function, wherein the step length of each layer of deconvolution is 2 and the padding is 1, and the output channel number is 256, 128, 64 and 32 in turn; the feature maps of each layer of the decoder are connected by channel splicing to realize the jump connection, and a 1*1 convolution kernel and a Softmax activation function are used in the output layer part, and a probability graph corresponding to the preset semantic label number is output, and the class corresponding to the maximum value of the probability of each pixel position is the semantic label of the pixel, which can avoid confusing the features of different regions.

[0092] Step 2: Obtain the multispectral reflectance curve of the standard material in the range of 400-1700 nm, compare it with the multispectral reflectance curve of each pixel of the textile to be detected, determine the reflectance deviation of each characteristic waveband, and mark the pixels with pollutants when the reflectance deviation of multiple characteristic wavebands exceeds the deviation threshold, and mark the remaining pixels as non-pollutant pixels, and simultaneously match the spectrum of the pollutant pixel with the known stain spectrum to output the stain type and pixel position;

[0093] The material classification of textiles is usually based on fiber composition. A piece of textile may be composed of a single material or a plurality of blended materials. High-spectral imaging technology is adopted, the spectral range is preset as 400-1700 nm, and the resolution is set as 10 nm. The range of 400-1700 nm covers visible light (400-700 nm), near-infrared short wave (700-1700 nm), which has the strongest spectral differentiation ability for textile materials and pollutants. A hyperspectral image data cube is constructed, the reflection light signal intensity of each pixel on the surface of the textile to be detected at each wavelength is directly obtained by a light splitting component and a detector, the average value of the reflection light signal intensity of all pixels is calculated, and a reflectance formula is constructed:

[0094]

[0095] In the formula, R represents reflectance, I yb represents the reflection light signal intensity of the standard white plate under the same light condition and wavelength, I bb represents the reflection light signal intensity of the textile to be detected at a certain wavelength.

[0096] The reflectivity curves of each standard material in the 400-1700nm wavelength band are obtained from the material spectral reference library. The spectral characteristics of each standard material are determined by multiple molecular groups. The characteristic wavelength band can most significantly distinguish a certain type of material from other materials or the specific wavelength point of the material itself and the contaminant. Therefore, based on the unique spectral characteristics of each standard material, the reflectivity of 5-8 characteristic wavelength bands of the reference sample is extracted to form a standard material-characteristic wavelength band-reflectivity control table, avoiding misjudgment of a single wavelength band. For example, cotton fiber contains a large number of hydroxyl groups, and the frequency multiplication and frequency combination of the vibration has multiple absorption peaks in the 700-1700nm wavelength band. The characteristic wavelength bands can be selected as 760nm, 960nm, 1190nm, 1450nm, 1550nm and 1650nm, and the corresponding reflectivity is extracted. In order to avoid errors of a single sample, the reflectivity of the characteristic wavelength band of multiple reference samples can be collected, and the average value is taken as the reflectivity corresponding to the characteristic wavelength band.

[0097] The reflectivity of each pixel at the characteristic wavelength band is compared with the corresponding characteristic wavelength band of all standard materials in the control table, and the matching degree is calculated.

[0098] d(i) = |R(i)-R'(i)|

[0099] In the formula, R(i) represents the reflectivity of a certain pixel of the textile to be detected at the ith characteristic wavelength band, R'(i) represents the reflectivity of a certain material at the ith characteristic wavelength band, and d(i) represents the matching degree of a certain pixel of the textile to be detected at the ith characteristic wavelength band. Wherein, i represents the serial number index of the characteristic wavelength band, i = 1, 2, …, n0, n0 represents the number of selected characteristic wavelength bands, 5≤n0≤8.

[0100] The average deviation is calculated based on the matching degree.

[0101]

[0102] In the formula, D represents the average deviation.

[0103] The deviation threshold D is set in advance TH , and D TH >0, when the calculated D of each standard material is greater than D TH , it is determined that the pixel is a contaminant pixel. Wherein, the deviation threshold can be set for each pixel of each sample, and the average deviation of each pixel from the control table is calculated. The average value plus 3 times the standard deviation is taken as the deviation threshold.

[0104] Obtain a published stain spectrum database, such as Sadtler spectrum database, covering spectrum data of polymers, dyes, contaminants, etc., and providing near-infrared and Raman spectrum support, highly matching the 400-1700nm spectrum range, and matching the full-band reflectivity curve of each contaminant pixel with the standard reflectivity curve of the stain spectrum database wave by wave, and calculating the spectral angle distance of the two.

[0105]

[0106] In the formula, R wu (i″) represents the reflectivity of the contaminant pixel at the i″th waveband, R uu (i″) represents the reflectivity of a certain type of stain in the stain spectrum database at the i″th characteristic waveband, θ i″ represents the spectral angle distance of the i″th waveband, i″ represents the serial number index of the full waveband, 400nm corresponds to i″=1, 410nm corresponds to i″=2, and 1700nm corresponds to i″=131, θ represents the spectral angle distance, the spectral angle distance is the angle between two spectral vectors, the smaller the value, the higher the spectral similarity, and the larger the value, the more significant the difference between the spectral curves;

[0107] When the spectral angle distance between a certain pixel and a certain type of stain in the database is less than a preset threshold, for example, 0.15, the type of the stain and the location of the pixel are output, wherein each pixel can be assigned a unique number, and when a contaminant pixel is detected, its corresponding number is directly recorded. Through a large number of actual detection experiments, the stains and the corresponding standard spectra in the database are concentrated in the 0-0.15 radian, and when the spectral angle distance of all stains is greater than the preset threshold, the type of the stain is output as unknown.

[0108] Table 1 shows that under each material, randomly selected pixels are detected to see if they are contaminant pixels, and the pixel type output table is generated after summarizing, which covers single materials such as cotton, polyester fiber, wool, silk, and blended materials such as cotton blend, hemp blend, and silk blend. By comparing the average deviation of each pixel with the deviation threshold of the corresponding material, it is determined whether the pixel is a contaminant, and the spectrum of the contaminant pixel is matched with the known stain spectrum to output the stain type, and the pixel type output table is generated after summarizing.

[0109] Table 1 Pixel Type Output Table

[0110]

[0111]

[0112] Step 3: convert the textile standard sample and the RGB image generated by the semantic label to be detected into the LAB color space, extract the rotated minimum bounding rectangle, evenly divide the grid matrix and sequentially number, fill the invalid pixels in the grid with the weighted average of the valid pixels in the neighborhood, calculate the average of the LAB channel in each grid, and fill the pollutant pixels according to their grid number position with the LAB channel average of the corresponding grid of the textile standard sample;

[0113] For the RGB image of the textile standard sample, based on the pre-labeled semantic label, the image sub-blocks of each semantic region are intercepted by the image segmentation algorithm. The specific process is as follows: mask extraction is performed on the RGB image, the pixels of the same semantic RGB image to be cropped are set to 255, and the rest are set to 0 to generate a binary image. The contour is extracted using a boundary extraction algorithm such as the Suzuki-Abe algorithm, which can extract the contour of all connected regions from the binary image, generating image sub-blocks that contain only pixel information of a single semantic region. The extracted semantic region sub-blocks are converted pixel by pixel using the color space conversion formula, and LAB color data is output, for example, using Adobe Photoshop image processing software to output LAB color data for each pixel. For the image sub-blocks of each semantic region intercepted based on the pre-labeled semantic label by the image segmentation algorithm, the pixel coordinate set is defined as where n1 represents the total number of pixels under this semantic label.

[0114] The average coordinates of all pixels are taken as the region center O(x c , y c ), and the centering process is performed on all pixel coordinates:

[0115]

[0116] In the formula, x i′ represents the horizontal coordinate after centering, y i′ represents the vertical coordinate after centering, x c represents the horizontal coordinate of a certain pixel, y c represents the vertical coordinate of a certain pixel, and i' represents the index of the pixel coordinate, i' = 1, 2, …, n1.

[0117] The principal axis slope k is obtained by the least squares method The rotation angle θ = arctan(k), the distance between the two pixels farthest apart on the principal axis is taken as the length of the rotated rectangle, and the distance between the two pixels farthest apart in the direction perpendicular to the principal axis is taken as the width of the rotated rectangle. With the center O as the origin, offset each half of the rotated rectangle length along the principal axis direction, and offset each half of the rotated rectangle width perpendicular to the principal axis direction as the four vertices, forming a rotated minimum bounding rectangle.

[0118] A local coordinate system is constructed with the center of the rotating minimum circumscribed rectangle as the origin, the main axis direction as the u-axis, and the vertical main axis direction as the v-axis. The rotating minimum circumscribed rectangle is uniformly divided into a grid matrix with dimensions a*b, so that the number of grid divisions of image sub-blocks in the same semantic region remains consistent, ensuring that subsequent grid numbering forms a one-to-one accurate association, completes the fast positioning of pollutant pixels in the textile standard sample and calls the corresponding filling value to provide a deterministic spatial index basis, and a, b>0, the boundary of the rectangle in the coordinate system is [u min , u max ]x[v min , v max ], wherein the size of the grid in the u-axis direction is:

[0119]

[0120] In the formula, s u represents the size of the grid in the u-axis direction;

[0121] The size of the grid in the v-axis direction is:

[0122]

[0123] In the formula, s v represents the size of the grid in the v-axis direction, the divided grid matrix is numbered in turn, and the numbering rule is to start from the first grid at the top left corner of the rotating minimum circumscribed rectangle, number from left to right in the u-axis direction, and continue to number from left to right in the horizontal sequence from the leftmost grid of the next row when the numbering of a row of grids is completed, until all grids are assigned unique numbers.

[0124] The edges of the textile semantic region are often continuous, and the following operations are independently performed for each grid:

[0125] Pixels outside the original semantic region are treated as invalid pixels. A 3*3 pixel square region is constructed with the invalid pixel P0(x0, y0) as the origin, which is the smallest symmetric unit. In textile semantic segmentation, invalid pixels are often close to the region boundary, and the closer the pixel is to the invalid pixel, the higher the reference value of the invalid pixel. In the square region, filter the pixels belonging to the current semantic region and record them as the effective pixel set E. For example, when the nearest neighbor of the invalid pixel is a red pattern region on the textile, the red feature of the pattern edge can be directly called through the 3*3 pixel square region to fill the color gap of the nearby invalid pixel. This processing can make the color transition of the semantic region edge more natural, avoid the subsequent sliding window from being misjudged as an abnormal region due to the sudden change of the LAB color space value at this position, and thus reduce unnecessary manual review workload;

[0126] When the 3*3 pixel square region has no valid pixels, then repeat the square neighborhood side length increase of 2 pixels, that is, expand 1 pixel up, down, left and right, until at least one valid pixel is found, and the target of symmetric expansion from the minimum neighborhood is achieved, and the valid pixels P in the final neighborhood are filled j (x j , y j ), based on the Manhattan distance calculation method, the pixel distance between the valid pixel and the invalid pixel is calculated:

[0127] d j = |x j -x0| + |y j -y0|

[0128] In the formula, d j represents the pixel distance from the valid pixel to the invalid pixel, j represents the index of the valid pixel, j = 1, 2, …, n2, n2 represents the number of valid pixels, and the Manhattan distance reflects the total pixel span in the horizontal and vertical directions, that is, the total number of grid movements, and the physical proximity between the valid pixel and the invalid pixel, thereby achieving the goal of quickly finding the nearest valid pixel and providing a data source for subsequent weight allocation;

[0129] The reciprocal of the pixel distance from the valid pixel to the invalid pixel is accumulated to obtain d total , and the weight of each valid pixel is calculated:

[0130]

[0131] In the formula, ω j represents the weight of each valid pixel;

[0132] The color of the near neighbor pixel is more representative of the characteristics of the invalid pixel, so the closer the valid pixel to the invalid pixel, the greater the influence on the fill value, so for the invalid pixel, the LAB channel is respectively calculated and filled with weighted calculation:

[0133]

[0134] In the formula, L0, A0, B0 represent the fill value of the invalid pixel, and L j , A j , B j represent the LAB three-channel values of the valid pixel. When the neighborhood part exceeds the boundary of the rotated rectangle, only the part within the rectangle is retained as the effective search range. For example, the grid on the right edge of the rectangle, the part on the right side of its neighborhood is ignored, and only the pixels on the left, above and below are searched. Then, the average value of the L, A and B channels in each grid is calculated to provide a color reference value for the subsequent calibration of the contaminant pixel position, and the filling of the corresponding grid contaminant pixel in the subsequent image to be detected is completed.

[0135] The L channel alone carries the brightness information, which is directly related to the light reflection characteristics of the material surface. The pilling area of the textile is rough due to fiber aggregation, and its brightness is significantly different from the normal area. By analyzing the brightness distribution of the L channel, the pilling area can be accurately located. The A and B channels are designed based on the visual characteristics of the human eye, and their numerical differences are more consistent with human perception of "color similarity". Compared with the RGB channel, which has a strong coupling between the three channels, resulting in "numerical similarity but visual difference" or "visual similarity but numerical dispersion", the A and B channel can cluster visually similar colors into a compact area, reducing the over-clustering phenomenon caused by strict numerical separation of RGB, ensuring that similar colors form a continuous semantic area, and reducing errors caused by differences in device lighting.

[0136] In the image generated by the same semantic label of the textile to be detected, the contaminant pixels need to be processed first: according to their grid number in the semantic area, locate the same grid number in the corresponding semantic area of the textile standard sample, extract the LAB mean value of this position to fill the contaminant pixels, and achieve the purpose of eliminating abnormal value interference before color space conversion. Contaminants such as stains and impurities have significant differences in spectral characteristics from the textile itself. If they are directly involved in subsequent color space conversion or grid analysis, it will cause the LAB channel data to deviate from the true material characteristics, forming false features, such as the high brightness of stains being misjudged as pilling.

[0137] After that, perform color space conversion on all non-contaminant pixels to obtain LAB images, divide the grid matrix according to the same rules as the textile standard sample to ensure that the grid space positions of the two correspond, and separately extract the L channel data of the converted image for subsequent pilling area detection. Finally, a complete LAB image is formed, with the same spatial position corresponding to the grid number, and the mean value of the corresponding area of the standard sample is filled, which can ensure that the filled value conforms to the normal color distribution rule of the area, and can also smooth local noise through the mean value of the standard sample, so that the filled image is consistent with the ideal clean sample in terms of spatial structure and color characteristics. When dividing the grid size, attention should be paid to the local feature scale of the semantic area, that is, the grid size should be close to the surface texture and pattern details of the textile. If the grid is too large, multiple textures or semantic areas will be mixed in one grid, making it difficult to distinguish local abnormalities, and the L channel mean and variance will be blurred. If the grid is too small, it will cause filling errors due to shooting or texture changes, and significantly increase the computational complexity. Experience suggests that a grid covers 16x16 or 32x32 pixel blocks, i.e. the grid matrix is divided into 16x16 or 32x32 grids. The grid size can be adjusted according to actual conditions.

[0138] Step 4: A window of a preset size is slid in the LAB color space of the same semantic label to extract the color features of the pixels in the window to complete fast clustering, and the standard deviation of each L channel is output under different clustering results to compare with a preset threshold to output a preliminary judgment of the pilling region;

[0139] A sliding window of w x w pixels is set, and for a 512 x 512 resolution image, the pilling diameter is about 8-20 pixels according to experience. To ensure that the window can completely contain the minimum defect feature, the value of w should be higher than 10 and lower than 30, so as to ensure that the window can completely contain the minimum defect feature, avoid the feature being segmented due to the window being too small, and avoid the window being too large to cover the local difference. The step is set to pixels, so that the adjacent windows overlap by 50%. For the case that the remaining pixels in the edge region are less than one window size, for horizontal sliding, when the current column start position plus the window width exceeds the right boundary of the semantic region, the right boundary of the window is fixed at the last column, and the position is taken as the termination position of the current window, and the window is no longer moved to the right. For vertical sliding, when the current row start position plus the window height exceeds the lower boundary of the semantic region, the bottom of the window is fixed at the last row, and the position is taken as the termination position of the current window, and the window is no longer moved downward. For the area covered by each window, the LAB three-channel feature values of all pixels in the window are extracted. In each window, only the A and B channels are retained as clustering features, that is, the A channel and B channel data are extracted separately to form a local feature matrix, and the size is w 2 x 2, for example, a 16 x 16 window corresponds to 256 rows x 2 columns, and each row corresponds to the A channel and B channel data of one pixel. The K-means algorithm is used, and the distance measurement of the difference between the A and B channel values of two pixels is the Euclidean distance:

[0140]

[0141] In the formula, L o represents the difference between the A and B channel values of two pixels, A1 and A2 represent the A channel values of two pixels, and B1 and B2 represent the B channel values of two pixels. The range of the clustering result value is set to 2-4, that is, the data is divided into 2 classes, 3 classes, and 4 classes. Each division completes a clustering, and the Silhouette Score profile coefficient algorithm is used to output the result. The K value with the maximum profile coefficient is selected as the final clustering number of the window.

[0142] For each sliding window, based on the A and B channel clustering results, the clustering cluster label to which each pixel belongs is obtained, for example, 1, 2, …, n3, and n3 represents the final clustering number. The standard deviation of the L channel value is calculated for each clustering cluster:

[0143]

[0144] In the formula, σ L represents the standard deviation of the L channel value of a cluster L, represents the L channel mean value of the cluster L, j″ represents the L channel value of the jth pixel of the cluster, j" = 1, 2,..., n4, n4 represents the number of pixels under the cluster;

[0145] A threshold value is preset. When the standard deviation is greater than the threshold value, it is considered that the pixels in the sliding window are the pilling region in the preliminary judgment. In most cases, the L channel of the smooth region of the fabric surface changes little in the smooth region, and the pilling defect will cause the gray scale in the window to fluctuate sharply. For example, a threshold value of 7 is selected for the preliminary judgment of pilling. A relatively low threshold value can also be selected to preferentially ensure the comprehensiveness of the potential pilling region. Even if a small amount of non-pilling region is included, it can also avoid missing the real pilling region due to the threshold value being too high. Since the marking is only used as the basis for positioning for subsequent refined modeling, the misjudged pilling region can be effectively eliminated.

[0146] Table 2 shows the number of clusters under different windows, and the standard deviation of the L channel of each cluster is compared with the threshold value to output the result of whether it is a pilling region in the preliminary judgment. After summarizing, 40 groups of data tables are generated,

[0147] Table 2 shows the preliminary judgment result of the pilling region

[0148]

[0149]

[0150] As Figure 2 shown in the figure, the figure shows the L channel standard deviation of different clusters in 40 sliding windows. The horizontal coordinate is the window number, the vertical coordinate is the L channel standard deviation, different colors represent different clusters, and the preset threshold value is used to judge whether it is a pilling region. The standard deviation of most clusters is concentrated in the 0-5 interval, indicating that the L channel value of most windows corresponding to the region fluctuates little, and the surface is relatively smooth. Some windows have a cluster standard deviation greater than 7. These windows may be pilling regions, so the position of the window should be preliminarily determined as a pilling region and should be paid attention to.

[0151] Step 5: For the pilling region in the preliminary judgment, a structured light imaging technology is used to obtain three-dimensional point cloud data, and the curvature feature in the region is calculated based on the three-dimensional point cloud data. When the curvature feature is consistent with the spherical feature, it is determined as a pilling region;

[0152] For example, an industrial-grade three-dimensional profile sensor such as the Keyence LJ-V7000 series, which integrates a laser emission module and a high-sensitivity imaging detection unit. Since the size of the pilling is usually 0.5-3mm, for the pilling area determined preliminarily, the Z-axis range is selected to be higher than 20mm in the measurement range, covering the pilling height and the fabric thickness fluctuation, and the X / Y-axis single measurement width is higher than 10mm, ensuring complete coverage of the pilling area determined preliminarily and its edge redundancy. In terms of precision, the Z-axis repeatability is selected to be lower than 0.01mm, and the X / Y-axis resolution is selected to be lower than 0.01mm, so as to distinguish the height difference of 0.5mm-level pilling and ensure the sampling density of the pilling edge details.

[0153] The device has built-in phase solving, absolute phase unwrapping and triangulation algorithms, which can realize the conversion from optical signal to three-dimensional coordinates and directly output three-dimensional point cloud data. In actual operation, the "high-speed profile mode" is selected, which is suitable for static detection scenarios. The laser type is blue laser, and the blue laser wavelength is 405nm, which can reduce the interference of the fabric surface reflection. The projection stripe width is limited to the ROI width, the horizontal sampling point number is set to 1400 points, and the vertical scanning frequency is set to 10kHz, so that the 0.5mm-sized pilling can also be sampled.

[0154] In the data acquisition stage, only the pilling area determined preliminarily is executed for laser projection and signal acquisition, and three consecutive acquisitions are performed to reduce random noise. The built-in software of the device automatically performs mean filtering processing on the three groups of data to finally generate a local point cloud containing X / Y / Z three-dimensional coordinates.

[0155] For the pilling area determined preliminarily, the local point cloud data is extracted:

[0156]

[0157] In the formula, Q represents the entire local point cloud data set, q m′ represents the m'th point of the point cloud data, (x m′ , y m′ , z m′ ) represents the coordinates of the m'th point of the point cloud data, N0 represents the total number of points in the point cloud data, m' represents the point cloud index, m' = 1, 2, …, N0;

[0158] For each point q m′ , based on the Euclidean distance with other points, the nearest 20 points in the space are selected to form a neighborhood point set Too few neighborhood points cannot accurately fit the surface, noise has a significant impact, and curvature calculation is unstable. Too many neighborhood points will contain too many geometric features, causing different curvature regions to cancel each other out. The selection of 20 points is an empirical value in the industrial detection scene to balance the calculation efficiency and accuracy. For the neighborhood point set Perform PCA, calculate the covariance matrix:

[0159]

[0160] where C m′ denotes the local covariance matrix of point q m′ , q denotes the center point of the neighborhood point set, q n′ denotes the neighborhood center point, where n' denotes the index of the point in the neighborhood point set, n' = 1, 2, …, 20,

[0161] Perform eigenvalue decomposition on the covariance matrix C m′ , and obtain three eigenvalues in ascending order: λ0, λ1, λ2, λ0≤λ1≤λ2, and construct the curvature calculation formula based on the eigenvalues:

[0162]

[0163] where ql denotes the curvature, and the covariance matrix describes the distribution characteristics of the neighborhood point set around the center point, which captures the characteristics of whether the point set is close to a spherical surface, a plane, a straight line, etc. by calculating the dispersion degree of the point set in each direction in three-dimensional space and the correlation between directions, λ0 reflects the dispersion degree of the point set in the "most compact" direction, λ1 reflects the dispersion degree of the point set in the second direction, and λ2 reflects the dispersion degree of the point set in the "most extended" direction. λ0+λ1+λ2 reflects the total dispersion of the point set in three directions, λ0 is the smallest eigenvalue, and the pucker is an approximate spherical convex of the fabric surface, and the geometric characteristics of the local three-dimensional point set are compact in all directions and significantly curved. Therefore, when the point set is close to a spherical surface, the dispersion degrees of the three directions are small, and the proportion of the minimum eigenvalue to the total dispersion is relatively small. The ratio of the two is essentially the ratio of the minimum dispersion to the total dispersion, thereby quantifying the bending degree of the point set. The larger the value, the greater the probability of the pucker region;

[0164] Then, for each point q m′ , only use its neighborhood point set to perform spherical fitting:

[0165] (x-a′) 2 +(y-b′) 2 +(z-c′) 2 =r 2

[0166] In the formula, (a', b', c') represents the three-dimensional coordinates of the sphere center, and r represents the spherical radius. The protrusions on the fabric surface can be pilling or other defects such as wrinkles, hair accumulation, stains, etc. The shapes of these non-pilling defects are usually not close to a sphere, for example, wrinkles are strip-shaped and hair is divergent. By fitting a sphere, the pilling can be distinguished from other defects in the shape dimension, reducing misjudgment.

[0167] Solve by minimizing the error objective function, where the formula for minimizing the error objective function is:

[0168]

[0169] Solve the optimal parameters (a', b', c', r) using the nonlinear least squares method, and for each point, calculate the sphericity:

[0170]

[0171] In the formula, S represents the sphericity. The point cloud of an ideal sphere is uniformly distributed in three-dimensional space, and the dispersion degrees in the three directions are small. At this time, λ0, λ1, and λ2 are close in value, and the proportional relationship tends to 1. The eigenvalues of the non-spherical shape are significantly different. When S << 1, it indicates that the point cloud is stretched or flattened in a certain direction, which does not conform to the spherical characteristics.

[0172] Calculate the fitting error:

[0173]

[0174] In the formula, E represents the fitting error, and x n′ , y n′ , and z n′ represent the three-dimensional coordinates of then'th point in the neighborhood point set . The essence is to calculate the average value of the square deviation of the distance from each point in the neighborhood to the fitting sphere center and the sphere radius. The smaller the value, the closer the actual point cloud distribution to the fitted sphere, which indirectly proves that the shape of the region conforms to the spherical characteristics of the pilling. Conversely, the larger the value, the greater the deviation between the fitted sphere and the actual point cloud, which may be caused by noise or non-pilling defects such as wrinkles. The formula has the significance of quantifying the degree of agreement between the neighborhood point set and the fitted sphere.

[0175] For each point q m′ , when ql>0.8, S>0.7, E<0.01, and 0.25≤r≤1.5 exist, it is considered that the preliminary judged pilling region belongs to the pilling region. These threshold values are based on the essential characteristics of pilling and are determined through statistical analysis of sample data and experimental verification, thereby realizing the cooperative screening of multi-dimensional features and effectively excluding noise to find a region that meets the shape protrusion, close to the sphere, regular smoothness, and reasonable size.

[0176] Table 3 shows the data collected under 40 groups of windows, based on three-dimensional point cloud data and determining whether it is a pilling area, after summarizing to generate a pilling area confirmation table.

[0177] Table 3 Pilling Area Confirmation Table

[0178]

[0179]

[0180] As shown in Figures 3-4 , by coordinating multiple indicators such as curvature, radius, sphericity, and fitting error, it can be determined whether it is a pilling area. For example, in the window number interval 15-30, the fitting error is reduced, the sphericity is greatly increased, from a low value to more than 7, even close to 10, the curvature value is obviously increased to close to or more than 0.8, and the radius is also concentrated in the interval of 0.3-0.7 mm, which is consistent with the characteristics of the sphere, so it is determined to be a pilling area.

[0181] Referring to Figure 5 , the application further provides a textile surface defect detection system based on image features, which is used to execute the above-mentioned textile surface defect detection method based on image features, comprising:

[0182] a semantic segmentation module, configured to acquire RGB images of a textile to be detected and a standard sample of the textile, after uniformly scaling the images to a resolution of 512*512 pixels, using a trained recognition model to perform semantic segmentation, and outputting semantic labels corresponding to each pixel;

[0183] a spectral detection module, configured to acquire a multi-spectral reflection curve of a standard material in a range of 400-1700 nm, compare the multi-spectral reflection curve with a multi-spectral reflectivity curve of each pixel of the textile to be detected, determine reflectivity deviations of each characteristic waveband, mark pixels with contaminants when the reflectivity deviations of multiple characteristic wavebands exceed a deviation threshold, mark the remaining pixels as non-contaminant pixels, and simultaneously match the spectrum of the contaminant pixels with known stain spectra, and output stain types and pixel positions;

[0184] a contaminated pixel filling module, configured to convert the standard sample of the textile and the RGB image generated by the semantic labels into a LAB color space, extract a rotated minimum bounding rectangle, uniformly divide a grid matrix and sequentially number the grid matrix after the extraction, fill invalid pixels in the grid with a weighted average of valid pixels in the neighborhood of the invalid pixels, calculate the average value of the LAB channel in each grid, and fill the contaminant pixels with the average value of the LAB channel of the corresponding grid of the standard sample of the textile according to the grid number position.

[0185] The pilling preliminary judgment module is configured to slide a preset size window in a LAB color space of the same semantic label, extract color features of pixels in the window to complete fast clustering, and output a standard deviation of each L channel under different clustering results, and compare the standard deviation with a preset threshold to output a preliminary judged pilling region;

[0186] The pilling confirmation module is configured to acquire three-dimensional point cloud data by using a structured light imaging technology for the preliminary judged pilling region, calculate a curvature feature in the region based on the three-dimensional point cloud data, and determine the region as a pilling region when the curvature feature is consistent with a spherical feature.

[0187] The above formulas are all dimensionless values, and the formulas are obtained by software simulation of a large amount of data to obtain a formula closest to the actual situation, and preset parameters in the formulas are set by a person skilled in the art according to actual conditions.

[0188] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized in the form of a computer program product wholly or partially. A person skilled in the art can realize that units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software methods depends on specific application and design constraints of the technical solutions.

[0189] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, and can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0190] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for detecting surface defects in textiles based on image features, characterized in that, The specific steps include: Step 1: Obtain RGB images of the textile to be detected and standard textile samples. After scaling the images to a uniform resolution of 512*512 pixels, use the trained recognition model to perform semantic segmentation and output the semantic label corresponding to each pixel. Step 2: Obtain the multispectral reflectance curve of the standard material in the range of 400-1700nm, compare it with the multispectral reflectance curve of each pixel of the textile to be tested, determine the reflectance deviation of each feature band, mark the pixel as a contaminant when the reflectance deviation of multiple feature bands exceeds the deviation threshold, and mark the rest as a pixel without contaminants. At the same time, match the spectrum of the contaminant pixel with the spectrum of known stains, and output the stain type and pixel location. Step 3: Convert the standard textile sample and the RGB image generated by the semantic tag of the textile to be detected into LAB color space. After extracting the minimum bounding rectangle of rotation, divide the grid matrix evenly and number it in sequence. Invalid pixels in the grid are filled with the weighted average of the valid pixels in their neighborhood. Calculate the average value of the LAB channel in each grid. According to the grid number position, the pollutant pixel is filled with the average value of the LAB channel of the corresponding grid of the standard textile sample. Step 4: Slide a window of a preset size in the LAB color space with the same semantic label, extract the color features of the pixels in the window to complete fast clustering, output the standard deviation of each L channel under different clustering results, and compare them with the preset threshold to output the preliminary judgment of the balling area. Step 5: For the initially identified balling area, use structured light imaging technology to acquire three-dimensional point cloud data. Calculate the curvature features within the area based on the three-dimensional point cloud data. If the curvature features match the spherical features, it is determined to be a balling area.

2. The method for detecting surface defects in textiles based on image features according to claim 1, characterized in that: Determining the reflectance deviation for each characteristic band, and marking pixels as contaminants when the reflectance deviations of multiple characteristic bands exceed a deviation threshold, includes the following steps: Using hyperspectral imaging technology, with a pre-set spectral range of 400-1700nm and a resolution of 10nm, a hyperspectral image data cube is constructed. The intensity of reflected light signal at each wavelength of each pixel on the surface of the textile to be tested is directly acquired through a beam splitter and detector. The average value of the reflected light signal intensity of all pixels is calculated. The reflectivity is calculated as the ratio of the intensity of reflected light signal at a certain wavelength of the textile to be tested to the intensity of reflected light signal at the same illumination and wavelength of a standard white board. The reflectance curves of each standard material in the 400-1700nm band were obtained from the material spectral reference library, and the reflectance of 5-8 characteristic bands of the reference sample were extracted based on the unique spectral characteristics of each standard material to form a reference material-characteristic band-reflectance comparison table. The reflectance of each pixel's characteristic band is compared with the corresponding characteristic bands of all standard materials in the reference table to calculate the matching degree. The matching degree is calculated as follows: the absolute value of the difference between the reflectance of a certain pixel of the textile to be tested and the reflectance of a certain standard material in the characteristic band. Then, the average deviation is calculated based on the matching degree, which is the average value of the matching degree of all characteristic bands. A deviation threshold is preset. When the average deviation calculated for each standard material is greater than the deviation threshold, the pixel is identified as a pollutant pixel.

3. The method for detecting surface defects in textiles based on image features according to claim 2, characterized in that: The method for matching the spectrum of contaminant pixels with the spectrum of known stains to output the stain type and pixel location is as follows: Obtain a publicly available stain spectral database, match the full-band reflectance curve of each contaminant pixel with the standard reflectance curve of the stain spectral database band by band, calculate the spectral angular distance between the two, and when the spectral angular distance between a pixel and a certain type of stain in the database is less than a preset threshold, output the type of stain and the location of the pixel.

4. The method for detecting surface defects in textiles based on image features according to claim 1, characterized in that: The method for converting standard textile samples and RGB images generated from semantic tags of the textiles to be detected into the LAB color space is as follows: For standard textile samples and RGB images of textiles to be tested, image segmentation algorithms are used to extract image sub-blocks of each semantic region based on pre-labeled semantic tags, so that each image sub-block contains only pixel information of a single semantic region. Color space conversion formulas are used to perform pixel-by-pixel conversion on each extracted semantic region sub-block, and LAB color data is output.

5. The method for detecting surface defects in textiles based on image features according to claim 4, characterized in that: The method for extracting the smallest bounding rectangle after rotation, uniformly dividing the mesh matrix, and numbering them sequentially is as follows: For image sub-blocks of semantic regions extracted using image segmentation algorithms based on pre-labeled semantic labels, the set of pixel coordinates for image sub-blocks of the same semantic region is defined as follows: Where n1 represents the total number of pixels under this semantic label; Take the average coordinates of all pixels as the region center O(x) c y c The coordinates of all pixels are centered: the x-coordinate after centering is the original x-coordinate minus the x-coordinate of the center of the region, and the y-coordinate after centering is the original y-coordinate minus the y-coordinate of the center of the region. The slope of the principal axis is obtained by the least squares method. The slope is calculated by summing the products of the x and y coordinates after all centering processes and dividing by the sum of the squares of the x coordinates after all centering processes. The rotation angle is the arctangent of the slope. The distance between the two farthest pixels on the principal axis is taken as the length of the rotation rectangle, and the distance between the two farthest pixels in the direction perpendicular to the principal axis is taken as the width of the rotation rectangle. With the center O as the origin, the length of the rotation rectangle is offset by half along the principal axis and the width of the rotation rectangle is offset by half in the direction perpendicular to the principal axis. These are used as the four vertices to form the minimum bounding rectangle of the rotation. A local coordinate system is constructed with the center of the smallest bounding rectangle of rotation as the origin, the principal axis as the u-axis, and the perpendicular direction of the principal axis as the v-axis. The smallest bounding rectangle of rotation is uniformly divided into a grid matrix of dimension a*b, ensuring that the number of grids in image sub-blocks with the same semantic region is consistent, and a, b>0. The boundary of the rectangle in this coordinate system is [u min u max ]×[v min v max ], where the mesh dimension in the u-axis direction is: In the formula, s u This indicates the size of the grid along the u-axis. The mesh dimension in the v-axis direction is: In the formula, s v This indicates the size of the mesh in the v-axis direction. The resulting mesh matrix is ​​numbered sequentially, starting from the top left corner of the smallest rotating bounding rectangle and proceeding from left to right along the u-axis. After completing the numbering of a row of meshes, the numbering continues from the leftmost mesh of the next row in a horizontal order until all meshes are assigned a unique number.

6. The method for detecting surface defects in textiles based on image features according to claim 5, characterized in that: The method for filling invalid pixels within a grid with the weighted average of the valid pixels in its neighborhood is as follows: Pixels outside the original semantic region are considered invalid pixels. A 3x3 pixel square region is constructed with the invalid pixel P0(x0, y0) as the origin. Pixels belonging to the current semantic region are selected within this square region and denoted as the valid pixel set E. When there are no valid pixels in the 3x3 pixel square region, the side length of the square neighborhood is repeatedly increased by 2 pixels (i.e., expanded by 1 pixel in all directions) until at least one valid pixel is found. The valid pixels P in the final neighborhood are then... j (x j y j ), calculate its pixel distance from the invalid pixel: d j =|x j -x0||y j -y0| In the formula, d j This represents the pixel distance from a valid pixel to an invalid pixel, where j represents the index of the valid pixel, j = 1, 2, ..., n2, and n2 represents the number of valid pixels; d is obtained by summing the reciprocals of the pixel distances from valid pixels to invalid pixels. total Calculate the weight of each valid pixel: In the formula, ω j This represents the weight of each valid pixel; For invalid pixels, perform weighted calculations and fill the LAB channels respectively: In the formula, L0, A0, and B0 represent the padding values ​​for invalid pixels, and L... j A j B j The LAB three-channel value represents the effective pixel.

7. The method for detecting surface defects in textiles based on image features according to claim 5, characterized in that: The method for quickly clustering pixels by sliding a window of a preset size within the LAB color space of the same semantic label and extracting color features from the pixels within the window is as follows: Set a w×w pixel sliding window and slide it through image sub-blocks of the same semantic region converted to LAB color space, setting the stride to [value missing]. For pixels, when the remaining pixels in the edge region are less than the size of a window, for horizontal sliding, when the starting position of the current column plus the window width exceeds the right boundary of the semantic region, the right boundary of the window is fixed at the last column, and this position is used as the termination position of the current window, and it no longer moves to the right. For vertical sliding, when the starting position of the current row plus the window height exceeds the lower boundary of the semantic region, the bottom edge of the window is fixed at the last row, and this position is used as the termination position of the current window, and it no longer moves down. For the area covered by each window, the LAB three-channel feature values ​​of all pixels in the window are extracted, and only the A and B channels are retained as clustering features. The Silhouette Score contour coefficient algorithm is used to output the clustering result value K.

8. The method for detecting surface defects in textiles based on image features according to claim 7, characterized in that: The method of outputting the standard deviation of each L channel under different clustering results and comparing it with the preset threshold to output the preliminary judgment of the balling area is as follows: Within the sliding window, for each clustering result, the L-channel values ​​of all pixels in each cluster are extracted. The standard deviation of the L-channel values ​​of each cluster is calculated. A threshold is preset. When the standard deviation is greater than the threshold, the pixels in the sliding window are considered as the preliminary balling area.

9. The method for detecting surface defects in textiles based on image features according to claim 1, characterized in that: The method for determining a region as a spherical region when the curvature features of the calculated region based on 3D point cloud data match the spherical features is as follows: To determine the actual location of the balling area based on preliminary assessment, three-dimensional point cloud data of the area was obtained using structured light imaging technology. In the formula, Q represents the entire local point cloud dataset, q m′ Let m' be the m-th point in the point cloud data, (x m′ y m′ , z m′ ) represents the coordinates of the m′-th point in the point cloud data, N0 represents the total number of points in the point cloud data, and m′ represents the point cloud index, m′=1,2,…,N0; For each point q m′ Select the 20 points that are closest to it in space to form a neighborhood point set. For neighborhood point set Perform PCA to calculate the covariance matrix: In the formula, C m′ Representing point q m′ The local covariance matrix, q represents the center point of the neighborhood point set. n′ Let n' represent the center point of the neighborhood, where n' represents the index of the set of neighborhood points, n' = 1, 2, ..., 20. For the covariance matrix C m′ Eigenvalue decomposition yields three eigenvalues ​​in ascending order: λ0, λ1, λ2, where λ0 ≤ λ1 ≤ λ2. Based on these eigenvalues, a formula for calculating curvature is constructed. In the formula, ql represents curvature; Then, for each point q m′ Only its neighborhood point set is used Perform spherical fitting: (x-a′) 2 +(y-b′) 2 +(z-c′) 2 =r 2 In the formula, (a′, b′, c′) represent the three-dimensional coordinates of the center of the sphere, and r represents the radius of the sphere. The solution is found by minimizing the objective function of the error, where the formula for minimizing the objective function is: The optimal parameters (a′, b′, c′, r) are solved using the nonlinear least squares method. For each point, the sphericity is calculated. In the formula, S represents sphericity; Calculate the fitting error: In the formula, E represents the fitting error, x n′ y n′ , z n′ Represents the neighborhood point set The three-dimensional coordinates of the n′-th point in the middle; For each point q m′ If ql>0.8, S>0.7, E<0.01, and 0.25≤r≤1.5, then the balling area in the preliminary judgment is considered to be a balling area.

10. A textile surface defect detection system based on image features, characterized in that: The system is used to perform the image feature-based textile surface defect detection method according to any one of claims 1-9: The semantic segmentation module is used to acquire RGB images of the textile to be detected and standard textile samples. After uniformly scaling the images to a resolution of 512*512 pixels, the trained recognition model is used to perform semantic segmentation and output the semantic label corresponding to each pixel. The spectral detection module is used to acquire the multispectral reflectance curve of the standard material in the range of 400-1700nm, compare it with the multispectral reflectance curve of each pixel of the textile to be tested, determine the reflectance deviation of each feature band, and mark the pixel as a contaminant when the reflectance deviation of multiple feature bands exceeds the deviation threshold, and mark the rest as a pixel without contaminants. At the same time, the spectrum of the contaminant pixel is matched with the spectrum of known stains, and the stain type and pixel location are output. The contamination pixel filling module is used to convert the standard textile sample and the RGB image generated by the semantic tag of the textile to be detected into the LAB color space. After extracting the minimum bounding rectangle of rotation, it uniformly divides the grid matrix and numbers it sequentially. Invalid pixels in the grid are filled with the weighted average of the valid pixels in their neighborhood. The average value of the LAB channel in each grid is calculated. The contamination pixel is filled with the average value of the LAB channel of the corresponding grid of the standard textile sample according to its grid number position. The initial balling detection module is used to slide a window of a preset size in the LAB color space with the same semantic label, extract the color features of the pixels in the window to complete fast clustering, output the standard deviation of each L channel under different clustering results, and compare them with the preset threshold to output the initial balling area. The balling confirmation module is used to acquire three-dimensional point cloud data using structured light imaging technology for the initially identified balling area. Based on the three-dimensional point cloud data, the curvature features within the area are calculated, and if they match the spherical features, the area is determined to be a balling area.

Images