Image feature-based textile surface defect detection method and system

Abstract

The invention provides a textile surface defect detection method and system based on image features, and relates to the technical field of textile surface defect detection.The method comprises the steps that an RGB image of a to-be-detected textile is obtained and zoomed to 512 * 512 pixels, and then at least five semantic tags are output through a semantic segmentation model; the method comprises the following steps: adopting hyperspectral imaging with the wavelength of 400-1700 nm and the resolution of 10 nm, calculating the pixel reflectivity through a reflectivity formula, comparing the reflectivity of 5-8 characteristic wavebands, marking as a pollutant if the average deviation exceeds a threshold value, matching the type and position of a stain with a spectral angular distance, and after a standard sample and a semantic region are converted into an LAB color space, extracting a minimum rotation enclosing rectangle and dividing a grid, invalid pixels are filled with 3 * 3 neighborhood effective pixel weighted mean values, and a grid LAB mean value is calculated. A grid pollutant pixel corresponding to an image to be detected is filled with a standard sample in a mean value mode, then channels A and B are clustered through a sliding window, a pilling area is preliminarily judged according to the standard deviation of a channel L, and the pilling area is confirmed by combining the structured light three-dimensional point cloud and comparing with a threshold value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of textile surface defect detection, in particular to a textile surface defect detection method and system based on image features. BACKGROUND

[0002] With the rapid development of large-scale production of the clothing industry and e-commerce retail, traditional manual detection for textile defect detection relies on experience judgment, which has strong subjectivity, low efficiency, high omission rate and other problems, and has been difficult to meet the needs of industrialized batch detection. In recent years, the application of computer vision has realized the automatic identification of textile quality. Through machine learning model analysis of clothing images, the positioning and classification of defects can be completed without human intervention, which significantly improves the detection efficiency and reduces the labor cost.

[0003] The existing technology for textile defect detection is mostly based on machine vision and deep learning. Through industrial cameras to collect clothing RGB images, deep learning models are used to extract image features to realize defect recognition and positioning. For example, through the color difference of the RGB image to identify obvious stains, or through the texture feature change to judge the pilling area. Some solutions will combine industrial-grade cameras to improve image resolution, or through multi-device linkage such as light source control system and conveyor belt optimization imaging environment to reduce external interference. However, the popularization and application of these technologies are still limited by multiple factors. On the one hand, the sensitivity to subtle defects is insufficient. RGB images can only capture two-dimensional color and texture information. For early-stage micro-pilling, subtle defects such as stains with a diameter of less than 1mm and light stains are easily hidden by background texture due to unobvious features, resulting in low detection rate. On the other hand, the method of global modeling not only requires expensive industrial high-resolution cameras, special light source systems and other equipment, but also has low efficiency, which is not suitable for e-commerce return detection, small and medium batch production quality inspection and other scenes.

[0004] Therefore, the industry urgently needs a method that can reduce the dependence on high-end hardware, adapt to diversified detection scenarios, and accurately model defects in different areas of clothing to improve the detection rate of subtle defects.

[0005] The above information disclosed in the background section is only used to enhance the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The purpose of the present application is to provide a textile surface defect detection method and system based on image features to solve the problems raised in the background.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] The method for detecting defects on a textile surface based on image features includes the following specific steps:

[0009] Step 1: Obtain the RGB images of the textile to be detected and the standard sample of the textile, uniformly scale the images to a resolution of 512*512 pixels, and then use the trained recognition model to perform semantic segmentation, outputting the semantic labels corresponding to each pixel.

[0010] Step 2: Obtain the multispectral reflectance curve of the standard material in the range of 400-1700 nm, compare it with the multispectral reflectance curve of each pixel of the textile to be detected, determine the reflectance deviation of each characteristic wavelength, and mark the pixels as contaminated when the reflectance deviation of multiple characteristic wavelengths exceeds the deviation threshold. The remaining pixels are marked as non-contaminated. Meanwhile, match the spectrum of the contaminated pixels with the known stain spectrum to output the stain type and pixel position.

[0011] Step 3: Convert the RGB images generated by the semantic labels of the standard sample of the textile and the textile to be detected into the LAB color space, extract the rotating minimum bounding rectangle, evenly divide the grid matrix and sequentially number them, fill the invalid pixels in the grid with the weighted average of the valid pixels in their neighborhood, calculate the mean value of the LAB channel in each grid, and fill the contaminated pixels with the mean value of the LAB channel of the corresponding grid of the standard sample of the textile according to their grid number position.

[0012] Step 4: Slide a window of a predetermined size in the LAB color space with the semantic labels, extract the color features of the pixels within the window to complete fast clustering, and output the standard deviation of each L channel under different clustering results. Compare the preliminary judged pilling area with the preset threshold.

[0013] Step 5: For the preliminary judged pilling area, use structured light imaging technology to obtain three-dimensional point cloud data, calculate the curvature features in the area based on the three-dimensional point cloud data, and determine the pilling area when the curvature features match the spherical features.

[0014] Further, determining the reflectance deviation of each characteristic wavelength and marking the pixels as contaminated when the reflectance deviation of multiple characteristic wavelengths exceeds the deviation threshold includes the following steps:

[0015] Using hyperspectral imaging technology, the spectral range is set to 400-1700 nm and the resolution is set to 10 nm. A hyperspectral image data cube is constructed, and the reflectance light signal intensity of each pixel on the surface of the textile to be detected at each wavelength is directly obtained through a light splitting component and a detector. The average value of the reflectance light signal intensity of all pixels is calculated. The calculation method of reflectance is as follows: the ratio of the reflectance light signal intensity of the textile to be detected at a certain wavelength to the reflectance light signal intensity of a standard white plate under the same lighting conditions and wavelength.

[0016] Obtain the reflectivity curve of each standard material in the 400-1700nm wave band from the material spectral reference library, and extract the reflectivity of 5-8 characteristic wave bands of the reference sample based on the unique spectral characteristics of each standard material, forming a standard material-characteristic wave band-reflectivity control table;

[0017] Compare the characteristic wave band reflectivity of each pixel with the corresponding characteristic wave band of all standard materials in the control table, and calculate the matching degree, wherein the calculation method of the matching degree is: the absolute value of the difference between the reflectivity of the characteristic wave band of a certain pixel of the textile to be detected and the reflectivity of the characteristic wave band of a certain standard material, and then calculate the average deviation based on the matching degree, that is, the average value of the matching degrees of all characteristic wave bands, and set a deviation threshold value, when the average deviation calculated under each standard material is greater than the deviation threshold value, the pixel is determined as a contaminant pixel.

[0018] Further, the method for outputting the stain type and pixel position by matching the spectrum of the contaminant pixel with the known stain spectrum is:

[0019] Obtain the published stain spectrum database, and match the full-wave band reflectivity curve of each contaminant pixel with the standard reflectivity curve of the stain spectrum database wave by wave, and calculate the spectral angle distance of the two, when the spectral angle distance of a certain pixel and a certain type of stain in the database is less than a preset threshold, output the type of the stain and the position of the pixel.

[0020] Further, the method for converting the textile standard sample and the RGB image generated by the semantic label into LAB color space is:

[0021] For the RGB image of the textile standard sample and the textile to be detected, based on the pre-labeled semantic label, the image sub-blocks of each semantic region are intercepted by the image segmentation algorithm, so that each image sub-block only contains pixel information of a single semantic region, and the extracted image sub-blocks of each semantic region are converted pixel by pixel using the color space conversion formula, and the LAB color data is output.

[0022] Further, the method for extracting a rotating minimum bounding rectangle and then uniformly dividing the grid matrix and sequentially numbering is:

[0023] For the image sub-blocks of each semantic region intercepted based on the pre-labeled semantic label by the image segmentation algorithm, for image sub-blocks of the same semantic region, define the pixel coordinate set of the image sub-blocks as Wherein, n1 represents the total number of pixels under this semantic label;

[0024] Take the average coordinates of all pixels as the region center O(x c , y c), all pixel coordinates are centralized, the centralized horizontal coordinate is the original horizontal coordinate minus the horizontal coordinate of the region center, and the centralized vertical coordinate is the original vertical coordinate minus the vertical coordinate of the region center;

[0025] The principal axis slope is obtained by the least square method, the slope is calculated by the sum of the product of all centralized horizontal coordinates and vertical coordinates divided by the sum of squares of all centralized horizontal coordinates, the rotation angle is the inverse tangent value of the slope, the distance between the two pixels on the principal axis is taken as the length of the rotated rectangle, the distance between the two pixels in the direction perpendicular to the principal axis is taken as the width of the rotated rectangle, the center O is taken as the origin, the rotated rectangle length is offset by half in the principal axis direction, and the rotated rectangle width is offset by half in the direction perpendicular to the principal axis, to form the four vertices of the rotated minimum bounding rectangle;

[0026] The local coordinate system is constructed with the center of the rotated minimum bounding rectangle as the origin, the principal axis direction as the u-axis, and the direction perpendicular to the principal axis as the v-axis, the rotated minimum bounding rectangle is uniformly divided into a grid matrix with dimensions a*b, the image sub-blocks of the same semantic region have consistent grid numbers, and a, b>0, the boundary of the rectangle in the coordinate system is [u min , u max ]x[v min , v max ], wherein the size of the grid in the u-axis direction is:

[0027]

[0028] In the formula, s u represents the size of the grid in the u-axis direction;

[0029] The size of the grid in the v-axis direction is:

[0030]

[0031] In the formula, s v represents the size of the grid in the v-axis direction, the divided grid matrix is sequentially numbered, the numbering rule is that the first grid in the upper left corner of the rotated minimum bounding rectangle is numbered from left to right in the u-axis direction, when the numbering of a row of grids is completed, the leftmost grid in the next row is numbered from left to right, and all grids are assigned unique numbers.

[0032] Further, the invalid pixels in the grid are filled by the weighted average of the valid pixels in the neighborhood as follows:

[0033] The pixels beyond the original semantic region are regarded as invalid pixels. A 3*3 pixel square region is constructed with the invalid pixel P0(x0, y0) as the origin. The pixels belonging to the current semantic region in the square region are screened and recorded as the valid pixel set E. When the 3*3 pixel square region has no valid pixel, the side length of the square neighborhood is increased by 2 pixels, i.e. the square neighborhood is expanded by 1 pixel upwards, downwards, leftwards and rightwards. This process is repeated until at least one valid pixel is found. The pixel distance between the valid pixel P j (x j , y j ) in the final neighborhood and the invalid pixel is calculated.

[0034] d j = |x j -x0||y j -y0|

[0035] In the formula, d j represents the pixel distance between the valid pixel and the invalid pixel, j represents the index of the valid pixel, j = 1, 2, …, n2, and n2 represents the number of valid pixels.

[0036] The reciprocal of the pixel distance between the valid pixel and the invalid pixel is accumulated to obtain d total , and the weight of each valid pixel is calculated.

[0037]

[0038] In the formula, ω j represents the weight of each valid pixel.

[0039] For the invalid pixel, the LAB channels are respectively executed to perform weighted calculation and filling.

[0040]

[0041] In the formula, L0, A0 and B0 represent the filling values of the invalid pixel, and L j , A j and B j represent the LAB three-channel values of the valid pixel.

[0042] Further, the window of the preset size is slid in the LAB color space of the same semantic label, and the color features of the pixels in the window are extracted to complete the method of fast clustering:

[0043] A sliding window of w*w pixels is set, and the sliding traversal is performed in the image sub-block of the same semantic region converted into the LAB color space. The step is set as For the case that there are not enough pixels for one window size for the edge area, for horizontal sliding, when the current column start position plus the window width exceeds the right boundary of the semantic area, the right boundary of the window is fixed at the last column, and the position is taken as the termination position of the current window, and no longer continue to move right; for vertical sliding, when the current row start position plus the window height exceeds the lower boundary of the semantic area, the bottom of the window is fixed at the last row, and the position is taken as the termination position of the current window, and no longer continue to move down; for the area covered by each window, the LAB three-channel feature values of all pixels in the window are extracted, only the A and B channels are reserved as clustering features, and the Silhouette Score profile coefficient algorithm is used to output the clustering result value K.

[0044] Further, the standard deviation of each L channel is output under different clustering results, and the method for outputting the preliminary judged pilling area by comparing the preset threshold value is:

[0045] In the sliding window, for each clustering result, the L channel values of all pixels in each clustering cluster are extracted, and the standard deviation of the L channel values of each clustering cluster is calculated. A threshold value is set in advance. When the standard deviation is greater than the threshold value, it is considered that the pixels of the sliding window are the preliminary judged pilling area.

[0046] Further, the method for determining the pilling area when the curvature feature in the region is consistent with the spherical feature is:

[0047] For the actual position of the preliminary judged pilling area, the three-dimensional point cloud data of the region is obtained by the structured light imaging technology:

[0048]

[0049] In the formula, Q represents the entire local point cloud data set, q m′ represents the m'th point of the point cloud data, (x m′ , y m′ , z m′ ) represents the coordinates of the m'th point of the point cloud data, N0 represents the total number of points in the point cloud data, m' represents the point cloud index, m' = 1, 2, …, N0;

[0050] For each point q m′ , select the nearest 20 points in its space to form a neighborhood point set

[0051] For the neighborhood point set , perform PCA to calculate the covariance matrix:

[0052]

[0053] In the formula, C m′Representing point q m′ The local covariance matrix, q represents the center point of the neighborhood point set. n′ Let n' represent the center point of the neighborhood, where n' represents the index of the set of neighborhood points, n' = 1, 2, ..., 20.

[0054] For the covariance matrix C m′ Eigenvalue decomposition yields three eigenvalues ​​in ascending order: λ0, λ1, λ2, where λ0 ≤ λ1 ≤ λ2. Based on these eigenvalues, a formula for calculating curvature is constructed.

[0055]

[0056] In the formula, ql represents curvature;

[0057] Then, for each point q m′ Only its neighborhood point set is used Perform spherical fitting:

[0058] (xa′) 2 +(yb′) 2 +(zc′) 2 =r 2

[0059] In the formula, (a′, b′, c′) represent the three-dimensional coordinates of the center of the sphere, and r represents the radius of the sphere.

[0060] The solution is found by minimizing the objective function of the error, where the formula for minimizing the objective function is:

[0061]

[0062] The optimal parameters (a′, b′, c′, r) are solved using the nonlinear least squares method. For each point, the sphericity is calculated.

[0063]

[0064] In the formula, S represents sphericity;

[0065] Calculate the fitting error:

[0066]

[0067] In the formula, E represents the fitting error, x n′ y n′ , z n′ Represents the neighborhood point set The three-dimensional coordinates of the n′-th point in the middle;

[0068] For each point q m′When ql>0.8, S>0.7, E<0.01, 0.25<=r<=1.5 exist, the pilling region of the preliminary judgment is considered to belong to the pilling region.

[0069] The application further provides an image feature-based textile surface defect detection system for executing the image feature-based textile surface defect detection method.

[0070] The semantic segmentation module is configured to acquire RGB images of the textile to be detected and a textile standard sample, uniformly scale the images to a resolution of 512*512 pixels, and perform semantic segmentation on the images using the trained recognition model to output semantic labels corresponding to each pixel.

[0071] The spectral detection module is configured to acquire a multispectral reflection curve of the standard material in a range of 400-1700 nm, compare the multispectral reflection curve with the multispectral reflection curve of each pixel of the textile to be detected, determine a reflection deviation of each characteristic waveband, mark a pixel as a contaminant pixel when the reflection deviation of the multiple characteristic wavebands exceeds a deviation threshold, mark the remaining pixels as non-contaminant pixels, and simultaneously match the spectrum of the contaminant pixel with a known stain spectrum to output a stain type and a pixel position.

[0072] The contaminated pixel filling module is configured to convert the textile standard sample and the RGB image generated based on the semantic labels into an LAB color space, extract a rotated minimum bounding rectangle, uniformly divide a grid matrix and sequentially number the grid matrix, fill invalid pixels in the grid matrix with a weighted average of valid pixels in the neighborhood of the invalid pixels, calculate an average value of the LAB channels in each grid, and fill the contaminant pixels with the average value of the LAB channels of the corresponding grid of the textile standard sample according to the grid number position of the contaminant pixels.

[0073] The pilling preliminary judgment module is configured to slide a window of a preset size in the LAB color space based on the semantic labels, extract color features of pixels in the window to complete fast clustering, output a standard deviation of each L channel under different clustering results, and compare the standard deviation with a preset threshold to output a preliminary pilling region.

[0074] The pilling confirmation module is configured to acquire three-dimensional point cloud data using a structured light imaging technology for the preliminary pilling region, calculate a curvature feature in the region based on the three-dimensional point cloud data, and determine the region as a pilling region when the curvature feature is consistent with a spherical feature.

[0075] Compared with the prior art, the application has the following advantages:

[0076] The present application is based on the division of different regions of the textile by semantic labels, the detection and positioning of stains are determined by hyperspectral imaging technology, and then the pixels determined as stains are filled according to the standard sample of the textile, so as to avoid the interference of the stain area on the subsequent brightness channel analysis, ensure that the brightness fluctuation characteristics of the pilling area can be accurately captured, and after the image of the textile to be detected is converted into LAB color space, a window of a preset size is slid, the standard deviation of each L channel under different clustering results is output based on the color characteristics, compared with the threshold value, and the suspected pilling part is preliminarily positioned, and then for the preliminarily positioned area, whether it is a pilling area is judged through three-dimensional point cloud data, the whole textile is not modeled in three dimensions, and the accuracy of pilling detection is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 It is a whole method flowchart of the present application;

[0078] Figure 2 It is an L channel standard deviation result graph of the clustering cluster of the present application;

[0079] Figure 3 It is a curvature and radius index graph of the present application;

[0080] Figure 4 It is a sphericity and fitting error index graph of the present application;

[0081] Figure 5 It is a whole system structure schematic diagram of the present application. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical scheme and advantages of the present application more clear and obvious, the present application is further described in detail below in combination with specific embodiments.

[0083] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present application should be understood as the usual meaning understood by those skilled in the art to which the present application belongs. The "first", "second" and similar words used in the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connected" or "connected" and similar words are not limited to physical or mechanical connection, but can include electrical connection, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent the relative positional relationship, when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0084] EMBODIMENT:

[0085] Please refer toFigures 1 to 4 The application provides a technical solution:

[0086] The image feature-based textile surface defect detection method comprises the following specific steps:

[0087] Step 1: Obtain the RGB images of the textile to be detected and the standard sample of the textile, uniformly scale the images to 512*512 pixel resolution, and then use the trained recognition model to perform semantic segmentation, and output the semantic labels corresponding to each pixel point;

[0088] DeepFashion2 Dataset is used as training data. The dataset contains a large number of textile images labeled with clothing categories, components and attributes, including 13 categories of clothing such as shirts, pants, dresses, coats and skirts. On this basis, the components of the clothes are further subdivided, such as collars, cuffs, pockets, waistbands and skirts. Each pixel is only labeled with one label. At least 1000 image samples covering multiple categories of textiles such as shirts, pants and socks are selected from the dataset. The pixel mask belongs to multi-class pixel-level labeling, for example, collar pixels, cuff pixels, etc. Without additional manual labeling, at least 5 preset semantic labels are output for each pixel, for example, for jeans, the pixels are labeled with waistband, leg, pocket, crotch and leg; for shirts, the pixels are labeled with body, collar, cuff, pocket, hem, shoulder, buttonhole, waistband;

[0089] Preprocess all images: retain the main body of the textile, uniformly crop to a square region in the zero-padded manner, and then scale to 512*512 pixel resolution;

[0090] Training process: the training batch size is set to 16, the cross-entropy loss function is used as the optimization objective, and the Adam optimizer is used, wherein the initial learning rate of the Adam optimizer is set to 0.001, the training is iterated on the training set of DeepFashion2 Dataset for 100 epochs, the learning rate is reduced to 1 / 10 of the original every 10 epochs, when the average intersection over union (mIoU) of the validation set provided by DeepFashion2 Dataset is higher than 0.85 for 5 consecutive epochs, it is considered to be stable, and the training is terminated in advance, if the condition is not met within 100 epochs, the maximum epoch is terminated;

[0091] The U-Ne semantic segmentation model is constructed: in the encoder part, 4 layers of convolution operation are adopted, each layer contains 2 times of 3*3 convolution, followed by ReLU activation after each convolution, and finally followed by 1 time of 2*2 maximum pooling, and the output channel number of the 4 layers of convolution is 64, 128, 256 and 512 in turn; in the decoder part, 4 layers of deconvolution operation are adopted, each layer contains a 3*3 transposed convolution kernel and a ReLU activation function, wherein the step length of each layer of deconvolution is 2 and the padding is 1, and the output channel number is 256, 128, 64 and 32 in turn; the feature maps of each layer of the decoder are connected by channel splicing to realize the jump connection, and a 1*1 convolution kernel and a Softmax activation function are used in the output layer part, and a probability graph corresponding to the preset semantic label number is output, and the class corresponding to the maximum value of the probability of each pixel position is the semantic label of the pixel, which can avoid confusing the features of different regions.

[0092] Step 2: Obtain the multispectral reflectance curve of the standard material in the range of 400-1700 nm, compare it with the multispectral reflectance curve of each pixel of the textile to be detected, determine the reflectance deviation of each characteristic waveband, and mark the pixels with pollutants when the reflectance deviation of multiple characteristic wavebands exceeds the deviation threshold, and mark the remaining pixels as non-pollutant pixels, and simultaneously match the spectrum of the pollutant pixel with the known stain spectrum to output the stain type and pixel position;

[0093] The material classification of textiles is usually based on fiber composition. A piece of textile may be composed of a single material or a plurality of blended materials. High spectral imaging technology is adopted, the spectral range is preset as 400-1700 nm, and the resolution is set as 10 nm. The range of 400-1700 nm covers visible light (400-700 nm), near-infrared short wave (700-1700 nm), which has the strongest spectral differentiation ability for textile materials and pollutants. A hyperspectral image data cube is constructed, the reflection light signal intensity of each pixel on the surface of the textile to be detected at each wavelength is directly obtained by a light splitting component and a detector, the average value of the reflection light signal intensity of all pixels is calculated, and a reflectance formula is constructed:

[0094]

[0095] In the formula, R represents reflectance, I yb represents the reflection light signal intensity of the standard white plate under the same light condition and wavelength, I bb represents the reflection light signal intensity of the textile to be detected at a certain wavelength.

[0096] The reflectivity curves of each standard material in the 400-1700nm wavelength band are obtained from the material spectral reference library. The spectral characteristics of each standard material are determined by multiple molecular groups. The characteristic wavelength band can most significantly distinguish a certain type of material from other materials or the specific wavelength point of the material itself and the contaminant. Therefore, based on the unique spectral characteristics of each standard material, the reflectivity of 5-8 characteristic wavelength bands of the reference sample is extracted to form a standard material-characteristic wavelength band-reflectivity control table, avoiding misjudgment of a single wavelength band. For example, cotton fiber contains a large number of hydroxyl groups, and the frequency multiplication and frequency combination of the vibration has multiple absorption peaks in the 700-1700nm wavelength band. The characteristic wavelength bands can be selected as 760nm, 960nm, 1190nm, 1450nm, 1550nm and 1650nm, and the corresponding reflectivity is extracted. In order to avoid errors of a single sample, the reflectivity of the characteristic wavelength band of multiple reference samples can be collected, and the average value is taken as the reflectivity corresponding to the characteristic wavelength band.

[0097] The reflectivity of each pixel at the characteristic wavelength band is compared with the corresponding characteristic wavelength band of all standard materials in the control table, and the matching degree is calculated.

[0098] d(i) = |R(i)-R'(i)|

[0099] In the formula, R(i) represents the reflectivity of a certain pixel of the textile to be detected at the ith characteristic wavelength band, R'(i) represents the reflectivity of a certain material at the ith characteristic wavelength band, and d(i) represents the matching degree of a certain pixel of the textile to be detected at the ith characteristic wavelength band. Wherein, i represents the serial number index of the characteristic wavelength band, i = 1, 2, …, n0, n0 represents the number of selected characteristic wavelength bands, 5≤n0≤8.

[0100] The average deviation is calculated based on the matching degree.

[0101]

[0102] In the formula, D represents the average deviation.

[0103] The deviation threshold D is set in advance TH , and D TH >0, when the calculated D of each standard material is greater than D TH , it is determined that the pixel is a contaminant pixel. Wherein, the deviation threshold can be set for each pixel of each sample, and the average deviation of each pixel from the control table is calculated. The average value plus 3 times the standard deviation is taken as the deviation threshold.

[0104] Obtain a published stain spectrum database, such as Sadtler spectrum database, covering spectrum data of polymers, dyes, contaminants, etc., and providing near-infrared and Raman spectrum support, highly matching the 400-1700nm spectrum range, and matching the full-band reflectivity curve of each contaminant pixel with the standard reflectivity curve of the stain spectrum database wave by wave, and calculating the spectral angle distance of the two.

[0105]

[0106] In the formula, R wu (i″) represents the reflectivity of the contaminant pixel at the i″th waveband, R uu (i″) represents the reflectivity of a certain type of stain in the stain spectrum database at the i″th characteristic waveband, θ i″ represents the spectral angle distance of the i″th waveband, i″ represents the serial number index of the full waveband, 400nm corresponds to i″=1, 410nm corresponds to i″=2, and 1700nm corresponds to i″=131, θ represents the spectral angle distance, the spectral angle distance is the angle between two spectral vectors, the smaller the value, the higher the spectral similarity, and the larger the value, the more significant the difference between the spectral curves;

[0107] When the spectral angle distance between a certain pixel and a certain type of stain in the database is less than a preset threshold, for example, 0.15, the type of the stain and the location of the pixel are output, wherein each pixel can be assigned a unique number, and when a contaminant pixel is detected, its corresponding number is directly recorded. Through a large number of actual detection experiments, the stains and the corresponding standard spectra in the database are concentrated in the 0-0.15 radian, and when the spectral angle distance of all stains is greater than the preset threshold, the type of the stain is output as unknown.

[0108] Table 1 shows that under each material, randomly selected pixels are detected to see if they are contaminant pixels, and the pixel type output table is generated after summarizing, which covers single materials such as cotton, polyester fiber, wool, silk, and blended materials such as cotton blend, hemp blend, and silk blend. By comparing the average deviation of each pixel with the deviation threshold of the corresponding material, it is determined whether the pixel is a contaminant, and the spectrum of the contaminant pixel is matched with the known stain spectrum to output the stain type, and the pixel type output table is generated after summarizing.

[0109] Table 1 Pixel Type Output Table

[0110]

[0111]

[0112] Step 3: convert the textile standard sample and the RGB image generated by the semantic label to be detected into the LAB color space, extract the rotated minimum bounding rectangle, evenly divide the grid matrix and sequentially number, fill the invalid pixels in the grid with the weighted average of the valid pixels in the neighborhood, calculate the average of the LAB channel in each grid, and fill the pollutant pixels according to their grid number position with the LAB channel average of the corresponding grid of the textile standard sample;

[0113] For the RGB image of the textile standard sample, based on the pre-labeled semantic label, the image sub-blocks of each semantic region are intercepted by the image segmentation algorithm. The specific process is as follows: mask extraction is performed on the RGB image, the pixels of the same semantic RGB image to be cropped are set to 255, and the rest are set to 0 to generate a binary image. The contour is extracted using a boundary extraction algorithm such as the Suzuki-Abe algorithm, which can extract the contour of all connected regions from the binary image, generating image sub-blocks that contain only pixel information of a single semantic region. The extracted semantic region sub-blocks are converted pixel by pixel using the color space conversion formula, and LAB color data is output, for example, using Adobe Photoshop image processing software to output LAB color data for each pixel. For the image sub-blocks of each semantic region intercepted based on the pre-labeled semantic label by the image segmentation algorithm, the pixel coordinate set is defined as where n1 represents the total number of pixels under this semantic label.

[0114] The average coordinates of all pixels are taken as the region center O(x c , y c ), and the centering process is performed on all pixel coordinates:

[0115]

[0116] In the formula, x i′ represents the horizontal coordinate after centering, y i′ represents the vertical coordinate after centering, x c represents the horizontal coordinate of a certain pixel, y c represents the vertical coordinate of a certain pixel, and i' represents the index of the pixel coordinate, i' = 1, 2, …, n1.

[0117] The principal axis slope k is obtained by the least squares method The rotation angle θ = arctan(k), the distance between the two pixels farthest apart on the principal axis is taken as the length of the rotated rectangle, and the distance between the two pixels farthest apart in the direction perpendicular to the principal axis is taken as the width of the rotated rectangle. With the center O as the origin, offset each half of the rotated rectangle length along the principal axis direction, and offset each half of the rotated rectangle width perpendicular to the principal axis direction as the four vertices, forming a rotated minimum bounding rectangle.

[0118] A local coordinate system is constructed with the center of the rotating minimum circumscribed rectangle as the origin, the main axis direction as the u-axis, and the vertical main axis direction as the v-axis. The rotating minimum circumscribed rectangle is uniformly divided into a grid matrix with dimensions a*b, so that the number of grid divisions of image sub-blocks in the same semantic region remains consistent, ensuring that subsequent grid numbering forms a one-to-one accurate association, completes the fast positioning of pollutant pixels in the textile standard sample and calls the corresponding filling value to provide a deterministic spatial index basis, and a, b>0, the boundary of the rectangle in the coordinate system is [u min , u max ]x[v min , v max ], wherein the size of the grid in the u-axis direction is:

[0119]

[0120] In the formula, s u represents the size of the grid in the u-axis direction;

[0121] The size of the grid in the v-axis direction is:

[0122]

[0123] In the formula, s v represents the size of the grid in the v-axis direction, the divided grid matrix is numbered in turn, and the numbering rule is to start from the first grid at the top left corner of the rotating minimum circumscribed rectangle, number from left to right in the u-axis direction, and continue to number from left to right in the horizontal sequence from the leftmost grid of the next row when the numbering of a row of grids is completed, until all grids are assigned unique numbers.

[0124] The edges of the textile semantic region are often continuous, and the following operations are independently performed for each grid:

[0125] Pixels outside the original semantic region are treated as invalid pixels. A 3*3 pixel square region is constructed with the invalid pixel P0(x0, y0) as the origin, which is the smallest symmetric unit. In textile semantic segmentation, invalid pixels are often close to the region boundary, and the closer the pixel is to the invalid pixel, the higher the reference value of the invalid pixel. In the square region, filter the pixels belonging to the current semantic region and record them as the effective pixel set E. For example, when the nearest neighbor of the invalid pixel is a red pattern region on the textile, the red feature of the pattern edge can be directly called through the 3*3 pixel square region to fill the color gap of the nearby invalid pixel. This processing can make the color transition of the semantic region edge more natural, avoid the subsequent sliding window from being misjudged as an abnormal region due to the sudden change of the LAB color space value at this position, and thus reduce unnecessary manual review workload;

[0126] When the 3*3 pixel square region has no valid pixels, then repeat the square neighborhood side length increase of 2 pixels, that is, expand 1 pixel up, down, left and right, until at least one valid pixel is found, and the target of symmetric expansion from the minimum neighborhood is achieved, and the valid pixels P in the final neighborhood are filled j (x j , y j ), based on the Manhattan distance calculation method, the pixel distance between the valid pixel and the invalid pixel is calculated:

[0127] d j = |x j -x0| + |y j -y0|

[0128] In the formula, d j represents the pixel distance from the valid pixel to the invalid pixel, j represents the index of the valid pixel, j = 1, 2, …, n2, n2 represents the number of valid pixels, and the Manhattan distance reflects the total pixel span in the horizontal and vertical directions, that is, the total number of grid movements, and the physical proximity between the valid pixel and the invalid pixel, thereby achieving the goal of quickly finding the nearest valid pixel and providing a data source for subsequent weight allocation;

[0129] The reciprocal of the pixel distance from the valid pixel to the invalid pixel is accumulated to obtain d total , and the weight of each valid pixel is calculated:

[0130]

[0131] In the formula, ω j represents the weight of each valid pixel;

[0132] The color of the near neighbor pixel is more representative of the characteristics of the invalid pixel, so the closer the valid pixel to the invalid pixel, the greater the influence on the fill value, so for the invalid pixel, the LAB channel is respectively calculated and filled with weighted calculation:

[0133]

[0134] In the formula, L0, A0, B0 represent the fill value of the invalid pixel, and L j , A j , B j represent the LAB three-channel values of the valid pixel. When the neighborhood part exceeds the boundary of the rotated rectangle, only the part within the rectangle is retained as the effective search range. For example, the grid on the right edge of the rectangle, the part on the right side of its neighborhood is ignored, and only the pixels on the left, above and below are searched. Then, the average value of the L, A and B channels in each grid is calculated to provide a color reference value for the subsequent calibration of the contaminant pixel position, and the filling of the corresponding grid contaminant pixel in the subsequent image to be detected is completed.

[0135] The L channel alone carries the brightness information, which is directly related to the light reflection characteristics of the material surface. The pilling area of the textile is rough due to fiber aggregation, and its brightness is significantly different from the normal area. By analyzing the brightness distribution of the L channel, the pilling area can be accurately located. The A and B channels are designed based on the visual characteristics of the human eye, and their numerical differences are more consistent with human perception of "color similarity". Compared with the RGB channel, which has a strong coupling between the three channels, resulting in "numerical similarity but visual difference" or "visual similarity but numerical dispersion", the A and B channel can cluster visually similar colors into a compact area, reducing the over-clustering phenomenon caused by strict numerical separation of RGB, ensuring that similar colors form a continuous semantic area, and reducing errors caused by differences in device lighting.

[0136] In the image generated by the same semantic label of the textile to be detected, the contaminant pixels need to be processed first: according to their grid number in the semantic area, locate the same grid number in the corresponding semantic area of the textile standard sample, extract the LAB mean value of this position to fill the contaminant pixels, and achieve the purpose of eliminating abnormal value interference before color space conversion. Contaminants such as stains and impurities have significant differences in spectral characteristics from the textile itself. If they are directly involved in subsequent color space conversion or grid analysis, it will cause the LAB channel data to deviate from the true material characteristics, forming false features, such as the high brightness of stains being misjudged as pilling.

[0137] After that, perform color space conversion on all non-contaminant pixels to obtain LAB images, divide the grid matrix according to the same rules as the textile standard sample to ensure that the grid space positions of the two correspond, and separately extract the L channel data of the converted image for subsequent pilling area detection. Finally, a complete LAB image is formed, with the same spatial position corresponding to the grid number, and the mean value of the corresponding area of the standard sample is filled, which can ensure that the filled value conforms to the normal color distribution rule of the area, and can also smooth local noise through the mean value of the standard sample, so that the filled image is consistent with the ideal clean sample in terms of spatial structure and color characteristics. When dividing the grid size, attention should be paid to the local feature scale of the semantic area, that is, the grid size should be close to the surface texture and pattern details of the textile. If the grid is too large, multiple textures or semantic areas will be mixed in one grid, making it difficult to distinguish local abnormalities, and the L channel mean and variance will be blurred. If the grid is too small, it will cause filling errors due to shooting or texture changes, and significantly increase the computational complexity. Experience suggests that a grid covers 16x16 or 32x32 pixel blocks, i.e. the grid matrix is divided into 16x16 or 32x32 grids. The grid size can be adjusted according to actual conditions.

[0138] Step 4: A window of a preset size is slid in the LAB color space of the same semantic label to extract the color features of the pixels in the window to complete fast clustering, and the standard deviation of each L channel is output under different clustering results to compare with a preset threshold to output a preliminary judgment of the pilling region;

[0139] A sliding window of w x w pixels is set, and for a 512 x 512 resolution image, the pilling diameter is about 8-20 pixels according to experience. To ensure that the window can completely contain the minimum defect feature, the value of w should be higher than 10 and lower than 30, so as to ensure that the window can completely contain the minimum defect feature, avoid the feature being segmented due to the window being too small, and avoid the window being too large to cover the local difference. The step is set to pixels, so that the adjacent windows overlap by 50%. For the case that the remaining pixels in the edge region are less than one window size, for horizontal sliding, when the current column start position plus the window width exceeds the right boundary of the semantic region, the right boundary of the window is fixed at the last column, and the position is taken as the termination position of the current window, and the window is no longer moved to the right. For vertical sliding, when the current row start position plus the window height exceeds the lower boundary of the semantic region, the bottom of the window is fixed at the last row, and the position is taken as the termination position of the current window, and the window is no longer moved downward. For the area covered by each window, the LAB three-channel feature values of all pixels in the window are extracted. In each window, only the A and B channels are retained as clustering features, that is, the A channel and B channel data are extracted separately to form a local feature matrix, and the size is w 2 x 2, for example, a 16 x 16 window corresponds to 256 rows x 2 columns, and each row corresponds to the A channel and B channel data of one pixel. The K-means algorithm is used, and the distance measurement of the difference between the A and B channel values of two pixels is the Euclidean distance:

[0140]

[0141] In the formula, L o represents the difference between the A and B channel values of two pixels, A1 and A2 represent the A channel values of two pixels, and B1 and B2 represent the B channel values of two pixels. The range of the clustering result value is set to 2-4, that is, the data is divided into 2 classes, 3 classes, and 4 classes. Each division completes a clustering, and the Silhouette Score profile coefficient algorithm is used to output the result. The K value with the maximum profile coefficient is selected as the final clustering number of the window.

[0142] For each sliding window, based on the A and B channel clustering results, the clustering cluster label to which each pixel belongs is obtained, for example, 1, 2, …, n3, and n3 represents the final clustering number. The standard deviation of the L channel value is calculated for each clustering cluster:

[0143]

[0144] In the formula, σ L represents the standard deviation of the L channel value of a cluster L, represents the L channel mean value of the cluster L, j″ represents the L channel value of the jth pixel of the cluster, j" = 1, 2,..., n4, n4 represents the number of pixels under the cluster;

[0145] A threshold value is preset. When the standard deviation is greater than the threshold value, it is considered that the pixels in the sliding window are the pilling region in the preliminary judgment. In most cases, the L channel of the smooth region of the fabric surface changes little in the smooth region, and the pilling defect will cause the gray scale in the window to fluctuate sharply. For example, a threshold value of 7 is selected for the preliminary judgment of pilling. A relatively low threshold value can also be selected to preferentially ensure the comprehensiveness of the potential pilling region. Even if a small amount of non-pilling region is included, it can also avoid missing the real pilling region due to the threshold value being too high. Since the marking is only used as the basis for positioning for subsequent refined modeling, the misjudged pilling region can be effectively eliminated.

[0146] Table 2 shows the number of clusters under different windows, and the standard deviation of the L channel of each cluster is compared with the threshold value to output the result of whether it is a pilling region in the preliminary judgment. After summarizing, 40 groups of data tables are generated,

[0147] Table 2 shows the preliminary judgment result of the pilling region

[0148]

[0149]

[0150] As Figure 2 shown in the figure, the figure shows the L channel standard deviation of different clusters in 40 sliding windows. The horizontal coordinate is the window number, the vertical coordinate is the L channel standard deviation, different colors represent different clusters, and the preset threshold value is used to judge whether it is a pilling region. The standard deviation of most clusters is concentrated in the 0-5 interval, indicating that the L channel value of most windows corresponding to the region fluctuates little, and the surface is relatively smooth. Some windows have a cluster standard deviation greater than 7. These windows may be pilling regions, so the position of the window should be preliminarily determined as a pilling region and should be paid attention to.

[0151] Step 5: For the pilling region in the preliminary judgment, a structured light imaging technology is used to obtain three-dimensional point cloud data, and the curvature feature in the region is calculated based on the three-dimensional point cloud data. When the curvature feature is consistent with the spherical feature, it is determined as a pilling region;

[0152] For example, an industrial-grade three-dimensional profile sensor such as the Keyence LJ-V7000 series, which integrates a laser emission module and a high-sensitivity imaging detection unit. Since the size of the pilling is usually 0.5-3mm, for the pilling area determined preliminarily, the Z-axis range is selected to be higher than 20mm in the measurement range, covering the pilling height and the fabric thickness fluctuation, and the X / Y-axis single measurement width is higher than 10mm, ensuring complete coverage of the pilling area determined preliminarily and its edge redundancy. In terms of precision, the Z-axis repeatability is selected to be lower than 0.01mm, and the X / Y-axis resolution is selected to be lower than 0.01mm, so as to distinguish the height difference of 0.5mm-level pilling and ensure the sampling density of the pilling edge details.

[0153] The device has built-in phase solving, absolute phase unwrapping and triangulation algorithms, which can realize the conversion from optical signal to three-dimensional coordinates and directly output three-dimensional point cloud data. In actual operation, the "high-speed profile mode" is selected, which is suitable for static detection scenarios. The laser type is blue laser, and the blue laser wavelength is 405nm, which can reduce the interference of the fabric surface reflection. The projection stripe width is limited to the ROI width, the horizontal sampling point number is set to 1400 points, and the vertical scanning frequency is set to 10kHz, so that the 0.5mm-sized pilling can also be sampled.

[0154] In the data acquisition stage, only the pilling area determined preliminarily is executed for laser projection and signal acquisition, and three continuous acquisitions are performed to reduce random noise. The built-in software of the device automatically performs mean filtering processing on the three groups of data to finally generate a local point cloud containing X / Y / Z three-dimensional coordinates.

[0155] For the pilling area determined preliminarily, the local point cloud data is extracted:

[0156]

[0157] In the formula, Q represents the entire local point cloud data set, q m′ represents the m'th point of the point cloud data, (x m′ , y m′ , z m′ ) represents the coordinates of the m'th point of the point cloud data, N0 represents the total number of points in the point cloud data, m' represents the point cloud index, m' = 1, 2, …, N0;

[0158] For each point q m′ , based on the Euclidean distance with other points, the nearest 20 points in the space are selected to form a neighborhood point set Too few neighborhood points cannot accurately fit the surface, noise has a significant impact, and curvature calculation is unstable. Too many neighborhood points will contain too many geometric features, causing different curvature regions to cancel each other out. The selection of 20 points is an empirical value in the industrial detection scene to balance the calculation efficiency and accuracy. For the neighborhood point set Perform PCA, calculate the covariance matrix:

[0159]

[0160] where C m′ denotes the local covariance matrix of point q m′ , q denotes the center point of the neighborhood point set, q n′ denotes the neighborhood center point, where n' denotes the index of the point in the neighborhood point set, n' = 1, 2, …, 20,

[0161] Perform eigenvalue decomposition on the covariance matrix C m′ , and obtain three eigenvalues in ascending order: λ0, λ1, λ2, λ0≤λ1≤λ2, and construct the curvature calculation formula based on the eigenvalues:

[0162]

[0163] where ql denotes the curvature, and the covariance matrix describes the distribution characteristics of the neighborhood point set around the center point, which captures the characteristics of whether the point set is close to a spherical surface, a plane, a straight line, etc. by calculating the dispersion degree of the point set in each direction in three-dimensional space and the correlation between directions, λ0 reflects the dispersion degree of the point set in the "most compact" direction, λ1 reflects the dispersion degree of the point set in the second direction, and λ2 reflects the dispersion degree of the point set in the "most extended" direction. λ0+λ1+λ2 reflects the total dispersion of the point set in three directions, λ0 is the smallest eigenvalue, and the pucker is an approximate spherical convex of the fabric surface, and the geometric characteristics of the local three-dimensional point set are compact in all directions and significantly curved. Therefore, when the point set is close to a spherical surface, the dispersion degrees of the three directions are small, and the proportion of the minimum eigenvalue to the total dispersion is relatively small. The ratio of the two is essentially the ratio of the minimum dispersion to the total dispersion, thereby quantifying the bending degree of the point set. The larger the value, the greater the probability of the pucker region;

[0164] Then, for each point q m′ , only use its neighborhood point set to perform spherical fitting:

[0165] (x-a′) 2 +(y-b′) 2 +(z-c′) 2 =r 2

[0166] In the formula, (a', b', c') represents the three-dimensional coordinates of the sphere center, and r represents the spherical radius. The protrusions on the fabric surface can be pilling or other defects such as wrinkles, hair accumulation, stains, etc. The shapes of these non-pilling defects are usually not close to a sphere, for example, wrinkles are strip-shaped and hair is divergent. By fitting a sphere, the pilling can be distinguished from other defects in the shape dimension, reducing misjudgment.

[0167] Solve by minimizing the error objective function, where the formula for minimizing the error objective function is:

[0168]

[0169] Solve the optimal parameters (a', b', c', r) using the nonlinear least squares method, and for each point, calculate the sphericity:

[0170]

[0171] In the formula, S represents the sphericity. The point cloud of an ideal sphere is uniformly distributed in three-dimensional space, and the dispersion degrees in the three directions are small. At this time, λ0, λ1, and λ2 are close in value, and the proportional relationship tends to 1. The eigenvalues of the non-spherical shape are significantly different. When S << 1, it indicates that the point cloud is stretched or flattened in a certain direction, which does not conform to the spherical characteristics.

[0172] Calculate the fitting error:

[0173]

[0174] In the formula, E represents the fitting error, and x n′ , y n′ , and z n′ represent the three-dimensional coordinates of then'th point in the neighborhood point set . The essence is to calculate the average value of the square deviation of the distance from each point in the neighborhood to the fitting sphere center and the sphere radius. The smaller the value, the closer the actual point cloud distribution to the fitted sphere, which indirectly proves that the shape of the region conforms to the spherical characteristics of the pilling. Conversely, the larger the value, the greater the deviation between the fitted sphere and the actual point cloud, which may be caused by noise or non-pilling defects such as wrinkles. The formula has the significance of quantifying the degree of agreement between the neighborhood point set and the fitted sphere.

[0175] For each point q m′ , when ql>0.8, S>0.7, E<0.01, and 0.25≤r≤1.5 exist, it is considered that the preliminary judged pilling region belongs to the pilling region. These threshold values are based on the essential characteristics of pilling and are determined through statistical analysis of sample data and experimental verification, thereby realizing the cooperative screening of multi-dimensional features and effectively excluding noise to find a region that meets the shape protrusion, close to the sphere, regular smoothness, and reasonable size.

[0176] Table 3 shows the data collected under 40 groups of windows, based on three-dimensional point cloud data and determining whether it is a pilling area, and after summarizing, a pilling area confirmation table is generated.

[0177] Table 3 Pilling Area Confirmation Table

[0178]

[0179]

[0180] As shown in Figures 3-4 , by coordinating multiple indicators such as curvature, radius, sphericity, and fitting error, it can be determined whether it is a pilling area. For example, in the window number interval 15-30, the fitting error is reduced, the sphericity is greatly increased, from a low value to more than 7, even close to 10, the curvature value is obviously increased to close to or more than 0.8, and the radius is also concentrated in the interval of 0.3-0.7 mm, which is consistent with the characteristics of the sphere, so it is determined to be a pilling area.

[0181] Referring to Figure 5 , the application further provides a textile surface defect detection system based on image features, which is used to execute the above-mentioned textile surface defect detection method based on image features, and comprises:

[0182] a semantic segmentation module, which is used to acquire RGB images of a textile to be detected and a standard sample of the textile, uniformly scale the images to a resolution of 512*512 pixels, and then use a trained recognition model to perform semantic segmentation, and output semantic labels corresponding to each pixel;

[0183] a spectrum detection module, which is used to acquire a multi-spectral reflection curve of a standard material in a range of 400-1700 nm, compare the multi-spectral reflection curve with a multi-spectral reflectivity curve of each pixel of the textile to be detected, determine reflectivity deviations of each characteristic waveband, mark pixels with contaminants when the reflectivity deviations of multiple characteristic wavebands exceed a deviation threshold, mark the remaining pixels as non-contaminant pixels, and simultaneously match the spectrum of the pixels with contaminants with known stain spectra, and output stain types and pixel positions;

[0184] a contaminated pixel filling module, which is used to convert the standard sample of the textile and the RGB image generated by the semantic labels of the textile to be detected into a LAB color space, extract a rotated minimum bounding rectangle, uniformly divide a grid matrix and sequentially number the grid matrix, fill invalid pixels in the grid with a weighted average of valid pixels in the neighborhood of the invalid pixels, calculate the average value of the LAB channel in each grid, and fill the pixels with contaminants with the average value of the LAB channel of the corresponding grid of the standard sample of the textile according to the grid number position.

[0185] The pilling preliminary judgment module is configured to slide a window of a preset size in a LAB color space of the same semantic label, extract color features of pixels in the window to complete fast clustering, and output a standard deviation of each L channel under different clustering results, and compare the standard deviation with a preset threshold to output a preliminary judged pilling region;

[0186] The pilling confirmation module is configured to acquire three-dimensional point cloud data by using a structured light imaging technology for the preliminary judged pilling region, calculate a curvature feature in the region based on the three-dimensional point cloud data, and determine the region as a pilling region when the curvature feature is consistent with a spherical feature.

[0187] The above formulas are all dimensionless values, and the formulas are obtained by software simulation of a large amount of data to obtain a formula closest to the actual situation, and preset parameters in the formulas are set by a person skilled in the art according to actual conditions.

[0188] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized in the form of a computer program product wholly or partially. A person skilled in the art can realize that units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software methods depends on specific application and design constraints of the technical solutions.

[0189] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, and can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0190] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for detecting surface defects of a textile based on image features, characterized in that, The specific steps include: Step 1: Obtain the RGB images of the textile to be detected and the textile standard sample, uniformly scale the images to 512*512 pixel resolution, and then use the trained recognition model for semantic segmentation to output the semantic label corresponding to each pixel point; Step 2: Obtain the multispectral reflectance curve of the standard material in the range of 400-1700nm, compare it with the multispectral reflectance curve of each pixel of the textile to be detected, determine the reflectance deviation of each characteristic waveband, and mark the pixels as contaminated when the reflectance deviation of multiple characteristic wavebands exceeds the deviation threshold; match the spectrum of the contaminated pixels with the known stain spectrum, and output the stain type and pixel position; Step 3: Convert the RGB images generated by the semantic labels of the textile standard sample and the textile to be detected to LAB color space, extract the rotating minimum bounding rectangle, evenly divide the grid matrix and sequentially number them, fill the invalid pixels in the grid with the weighted average of the valid pixels in their neighborhood, calculate the mean value of the LAB channel in each grid, and fill the contaminated pixels with the mean value of the LAB channel of the corresponding grid of the textile standard sample according to their grid number position; Step 4: Slide the window of a predetermined size in the LAB color space with the same semantic label, extract the color features of the pixels in the window to complete fast clustering, and output the standard deviation of each L channel under different clustering results to output the initial judgment of the pilling area; Step 5: For the initial judgment of the pilling area, use structured light imaging technology to obtain three-dimensional point cloud data, calculate the curvature features in the area based on the three-dimensional point cloud data, and determine the pilling area when the curvature features match the spherical features.

2. The image feature based textile surface defect detection method according to claim 1, characterized in that: The steps of determining the reflectance deviation of each characteristic waveband and marking the pixels as contaminated when the reflectance deviation of multiple characteristic wavebands exceeds the deviation threshold include: Using hyperspectral imaging technology, the spectral range is set to 400-1700nm and the resolution is set to 10nm, a hyperspectral image data cube is constructed, the reflectance light signal intensity of each pixel point on the surface of the textile to be detected at each wavelength is directly obtained through the light splitting component and the detector, the average value of the reflectance light signal intensity of all pixel points is calculated, and the calculation method of reflectance is: the ratio of the reflectance light signal intensity of the textile to be detected at a certain wavelength to the reflectance light signal intensity of the standard white plate under the same lighting conditions and wavelength; Obtain the reflectance curve of each standard material in the 400-1700nm waveband from the material spectrum reference library, and extract the reflectance of 5-8 characteristic wavebands of the reference sample based on the unique spectral characteristics of each standard material to form a standard material-characteristic waveband-reflectance control table; The feature band reflectivity of each pixel is compared with the corresponding feature band of all standard materials in the control table, and the matching degree is calculated, wherein the calculation method of the matching degree is: the absolute value of the difference between the reflectivity of the feature band of a pixel of the textile to be detected and the reflectivity of the feature band of a standard material, and then the average deviation is calculated based on the matching degree, that is, the average value of the matching degrees of all feature bands, and a deviation threshold is preset, when the average deviation calculated under each standard material is greater than the deviation threshold, the pixel is determined as a contaminant pixel.

3. The image feature based textile surface defect detection method according to claim 2, characterized in that: The method for matching the spectrum of the contaminant pixel with the known stain spectrum and outputting the stain type and pixel position is: An open stain spectrum database is obtained, the full-band reflectivity curve of each contaminant pixel is matched with the standard reflectivity curve of the stain spectrum database wave by wave, and the spectral angle distance of the two is calculated, when the spectral angle distance between a pixel and a certain type of stain in the database is less than a preset threshold, the type of the stain and the position of the pixel are output.

4. The image feature based textile surface defect detection method of claim 1, wherein: The method for converting the RGB images of the textile standard sample and the textile to be detected generated by the semantic label into LAB color space is: For the RGB images of the textile standard sample and the textile to be detected, based on the pre-labeled semantic label, the image sub-blocks of each semantic region are intercepted by an image segmentation algorithm, so that each image sub-block only contains pixel information of a single semantic region, and each semantic region sub-block is converted pixel by pixel by using a color space conversion formula, and LAB color data is output.

5. The image feature based textile surface defect detection method of claim 4, wherein: The method for extracting a rotated minimum bounding rectangle, uniformly dividing the grid matrix, and sequentially numbering is: For each semantic region image sub-block intercepted by the image segmentation algorithm based on the pre-labeled same semantic label, for the same semantic region image sub-block, define its pixel coordinate set as Wherein, n1 represents the total number of pixels under this semantic label; Take the average coordinate of all pixels as the region center O(x c , y c ), and center all pixel coordinates: the centered horizontal coordinate is the original horizontal coordinate minus the horizontal coordinate of the region center, and the centered vertical coordinate is the original vertical coordinate minus the vertical coordinate of the region center; The slope of the principal axis is obtained by the least square method, wherein the calculation process of the slope is that the sum of the products of all centralized horizontal coordinates and vertical coordinates is divided by the sum of the squares of all centralized horizontal coordinates, and the rotation angle is the inverse tangent value of the slope, the distance between the two pixels farthest from the principal axis is taken as the length of the rotated rectangle, the distance between the two pixels farthest in the direction perpendicular to the principal axis is taken as the width of the rotated rectangle, the center O is taken as the origin, the rotated rectangle length is offset by half in the direction of the principal axis, and the rotated rectangle width is offset by half in the direction perpendicular to the principal axis, to form a rotated minimum bounding rectangle. A local coordinate system is constructed with the center of the rotating minimum circumscribed rectangle as the origin, the main axis direction as the u-axis, and the vertical main axis direction as the v-axis. The rotating minimum circumscribed rectangle is uniformly divided into a grid matrix with dimensions a*b, so that the number of divided grids of image sub-blocks of the same semantic region remains consistent, and a, b>0. The boundary of the rectangle in the coordinate system is [u min , u max ]×[v min , v max ], wherein the size of the grid in the u-axis direction is: In the formula, s u represents the size of the grid in the u-axis direction; The size of the grid in the v-axis direction is: In the formula, s v The size of the grid in the v-axis direction is represented, and the divided grid matrix is numbered in turn. The numbering rule is that the numbering starts from the first grid at the upper left corner of the smallest circumscribed rectangle, and the numbering is in the u-axis direction from left to right. When the numbering of a row of grids is completed, the leftmost grid of the next row is continued to be numbered in the horizontal order, and all grids are assigned a unique number.

6. The image feature based textile surface defect detection method of claim 5, wherein: The invalid pixels in the grid are filled with the weighted average of the valid pixels in their neighborhood. The pixels beyond the original semantic region are regarded as invalid pixels. A 3*3 pixel square region is constructed with the invalid pixel P0(x0, y0) as the origin. The pixels belonging to the current semantic region in the square region are screened and recorded as the valid pixel set E. When the 3*3 pixel square region has no valid pixel, the side length of the square neighborhood is increased by 2 pixels, that is, each of the up, down, left and right is expanded by 1 pixel. This is repeated until at least one valid pixel is found. The pixel distance between the valid pixel P j (x j , y j ) in the final neighborhood and the invalid pixel is calculated. d j =|x j -x0||y j -y0| where d j represents the pixel distance from the valid pixel to the invalid pixel, j represents the index of the valid pixel, j = 1, 2, …, n2, and n2 represents the number of valid pixels. The reciprocal of the pixel distance from the valid pixel to the invalid pixel is accumulated to obtain d total The weight of each valid pixel is calculated: ωi j denotes the weight of each valid pixel; For invalid pixels, weighted calculation is performed on the LAB channels: In the formula, L0, A0, B0 represent the filling values of the invalid pixels, and L j , A j , B j represent the LAB three-channel values of the valid pixels.

7. The image feature based textile surface defect detection method of claim 5, wherein: The method for extracting the color features of the pixels in the window to complete fast clustering is: A sliding window of w x w pixels is set, and the image sub-blocks in the same semantic region converted into the LAB color space are traversed in sliding mode, with a step size of For the case where there are not enough window sizes for the remaining pixels in the edge region, for horizontal sliding, when the current column start position plus the window width exceeds the right boundary of the semantic region, the right boundary of the window is fixed at the last column, and this position is taken as the termination position of the current window, and no further right movement is performed. For vertical sliding, when the current row start position plus the window height exceeds the lower boundary of the semantic region, the bottom edge of the window is fixed at the last row, and this position is taken as the termination position of the current window, and no further downward movement is performed. For the region covered by each window, the LAB three-channel feature values of all pixels in the window are extracted, only the A and B channels are retained as clustering features, and the Silhouette Score profile coefficient algorithm is used to output the clustering result value K.

8. The image feature based textile surface defect detection method of claim 7, wherein: The standard deviation of each L channel is output under different clustering results, and the preliminary judged pilling area is output by comparing the preset threshold value: For each clustering result in the sliding window, the L channel values of all pixels in each clustering cluster are extracted, and the standard deviation of the L channel values of each clustering cluster is calculated, and a threshold value is preset, when the standard deviation is greater than the threshold value, it is considered that the pixels of the sliding window are the preliminary judged pilling area.

9. The image feature based textile surface defect detection method of claim 1, wherein: The method for calculating the curvature features in the region based on three-dimensional point cloud data and determining the pilling area when the curvature features are consistent with the spherical features is: For the actual position of the pilling region preliminarily judged, three-dimensional point cloud data of the region is acquired through a structured light imaging technology: wherein Q represents the entire local point cloud dataset, q m′ represents the m'th point of the point cloud data, (x m′ , y m′ , z m′ ) represents the coordinates of the m'th point of the point cloud data, N0represents the total number of points in the point cloud data, m' represents the point cloud index, m' = 1, 2,..., N0; For each point q m′ , the nearest 20 points in its space are selected to form the neighborhood point set A set of neighborhood points Perform PCA, compute covariance matrix: where C m′ denotes the local covariance matrix of point q m′ , denotes the center point of the neighborhood point set, q n′ denotes the neighborhood center point, where n' denotes the index of the point in the neighborhood point set, n' = 1, 2,..., 20, Eigenvalue decomposition is performed on the covariance matrix C m′ to obtain three eigenvalues λ0, λ1, λ2 arranged in ascending order: λ0≤λ1≤λ2. A curvature calculation formula is constructed based on the eigenvalues: In the formula, ql represents the curvature; After that, for each point q m′ , only using its neighborhood point set spherical fitting is performed: (x - a') 2 + (y - b') 2 + (z - c') 2 = r 2 In the formula, (a', b', c') represents the three-dimensional coordinates of the sphere center, and r represents the spherical radius; Solve by minimizing the error objective function, wherein the formula for minimizing the error objective function is: Solve the optimal parameters (a', b', c', r) using a nonlinear least squares method, and for each point, calculate the sphericity: In the formula, S represents the sphericity; Calculate the fitting error: where E represents the fitting error, x n′ , y n′ , and z n′ represent the three-dimensional coordinates of then'th point in the neighborhood point set . for each point q m′ When ql> 0.8, S> 0.7, E< 0.01, 0.25≤ r≤ 1.5 exist, then the initial judgment of the pilling region is considered to belong to the pilling region.

10. A system for detecting surface defects in a textile based on image features, characterized in that: The system is used to perform the image feature-based textile surface defect detection method of any one of claims 1-9: The semantic segmentation module is configured to acquire RGB images of the textile to be detected and a standard sample of the textile, uniformly scale the images to a resolution of 512*512 pixels, and perform semantic segmentation using a trained recognition model to output semantic labels corresponding to each pixel. The spectral detection module is configured to acquire a multi-spectral reflectance curve of the standard material in a range of 400-1700 nm, compare the multi-spectral reflectance curve of each pixel of the textile to be detected with the multi-spectral reflectance curve, determine the reflectance deviation of each characteristic waveband, mark pixels with contaminants when the reflectance deviation of multiple characteristic wavebands exceeds a deviation threshold, mark the remaining pixels as non-contaminant pixels, and simultaneously match the spectrum of the contaminant pixels with known stain spectra to output the stain type and pixel position. The contaminated pixel filling module is configured to convert the standard sample of the textile and the RGB image generated from the semantic labels into the LAB color space, extract a rotated minimum bounding rectangle, uniformly divide the grid matrix and sequentially number the grid matrix, fill the invalid pixels in the grid with a weighted average of the valid pixels in the neighborhood, calculate the average value of the LAB channel in each grid, and fill the contaminant pixels with the average value of the LAB channel of the corresponding grid of the standard sample of the textile according to the grid number position. The pilling preliminary judgment module is configured to slide a window of a predetermined size in the LAB color space with the semantic labels, extract the color features of the pixels in the window to complete fast clustering, output the standard deviation of each L channel under different clustering results, and compare the standard deviation with a predetermined threshold to output the pilling region preliminarily judged. The pilling confirmation module is configured to acquire three-dimensional point cloud data for the pilling region preliminarily judged using a structured light imaging technology, calculate the curvature features in the region based on the three-dimensional point cloud data, and determine the pilling region when the curvature features are consistent with the spherical features.

Images