Cloth flaw detection method based on high-frequency feature constraint and edge entropy minimization

Abstract

The invention relates to a cloth flaw detection method based on high-frequency feature constraint and edge entropy minimization, and the method comprises the steps: 1, collecting a flawless and abnormal cloth training picture data set, carrying out the initialization of hyper-parameters, and carrying out the initialization and fine adjustment of model parameters; 2, randomly sampling an image sample x, obtaining vector quantized image features through a feature encoder, generating flaw noise through an anomaly generator, adding the flaw noise into original features of an image, respectively inputting features with anomalies into an image reconstruction decoder and an image restoration decoder, and carrying out image reconstruction; and reconstructing an abnormal image and an abnormal image respectively, and the like. The detection method has the advantages that the high-frequency feature constraint module and the edge entropy minimization module are introduced, so that the cloth defect anomaly detection accuracy is remarkably improved; and the high-frequency feature constraint module converts the real image and the reconstructed image into frequency domain representation by using discrete Fourier transform.

Description

Technical Field

[0001] The present invention relates to a cloth image defect anomaly detection technology, and in particular to a cloth defect detection method with high-frequency feature constraints and edge entropy minimization. Background Art

[0002] In industrial production, the control and improvement of product quality has always been one of the core goals. Especially in the modern industrial environment, high-quality products not only determine the market competitiveness of enterprises, but also directly affect the economic benefits and brand reputation of enterprises.

[0003] In the textile industry, the quality of cloth directly affects the quality and market competitiveness of downstream products. Therefore, the detection of abnormal defects on the surface of cloth has become a key link in the textile industry. However, in the process of cloth production, due to long-term operation or operational errors, the machinery and equipment will inevitably fail or make mistakes, resulting in quality problems in the produced cloth. For example, holes, stains, pilling and color difference in the cloth, these abnormal defects will have an adverse effect on the appearance quality and performance of the cloth, and bring direct economic losses to the manufacturing company. Therefore, real-time abnormal defect detection of cloth has become a crucial link in the production process. Traditionally, textile companies usually use manual detection to monitor abnormal defects of cloth. Although this method is direct, it has significant limitations. First, manual detection requires a lot of human resources, has low detection efficiency, and is difficult to adapt to the needs of large-scale production. Secondly, the accuracy of manual detection is easily affected by the subjective factors of the inspectors. As the working time increases, the accuracy and consistency of the detection will gradually decrease. In addition, manual detection is difficult to accurately detect small and complex defects, especially in the production of cloth with high precision requirements, the effectiveness of this method is even more insufficient. Therefore, the development of efficient, accurate and automated cloth defect and anomaly detection technology has become a major issue that needs to be urgently addressed in the current textile industry.

[0004] In the context of modern industrial automation and intelligent manufacturing, the use of computer vision and artificial intelligence technology to achieve automated defect anomaly detection of cloth has become an important means and development trend. Computer vision technology obtains cloth image data through camera equipment and uses image processing and pattern recognition algorithms for analysis, which can effectively detect defects and anomalies on the surface of cloth. However, traditional computer vision methods mainly rely on manually designed feature extraction, which is often difficult to cope with different types of defect anomalies in the face of complex and changing industrial environments. In recent years, with the rapid development of deep learning technology, image processing algorithms based on deep learning have made significant progress in the field of computer vision. Deep learning can automatically learn high-level features in images through the training of multi-layer neural networks, significantly improving the performance of image classification, object detection and image segmentation. Especially in the field of unsupervised learning, deep learning algorithms have shown great potential, and can achieve effective feature learning and defect anomaly detection without a large amount of labeled data.

[0005] Existing unsupervised image defect anomaly detection methods mainly include algorithms based on feature embedding, simulation synthesis and image reconstruction. The feature embedding-based method uses convolutional neural networks to extract high-level features from images, maps these features to a low-dimensional embedding space, and detects anomalies by measuring the degree of deviation of features in the embedding space. The simulation synthesis method trains the defect anomaly detection model by introducing anomalies on normal data or generating synthetic data. The image reconstruction-based method is currently the most widely used one. It identifies abnormal areas by the difference in reconstruction ability of non-abnormal images, and is often implemented using models such as autoencoders and generative adversarial networks. Although the defect anomaly detection methods based on image reconstruction perform well on general data sets, in specific industrial scenarios, the performance of these methods is significantly reduced because cloth images usually have more details, textures and structures. The detailed texture features in cloth images are often contained in high-frequency information, while neural networks tend to fit low-frequency information during training, resulting in the loss of some fine-grained information. In addition, the high quality standards of cloth require that the defect anomaly detection model can accurately identify tiny defects, which places higher requirements on the accuracy of the detection model. The uncertainty of existing methods in predicting the edges of defect anomaly areas further limits their application in cloth image defect anomaly detection.

[0006] In view of the above problems, the present invention proposes a cloth defect detection method with high-frequency feature constraints and edge entropy minimization.

[0007] At present, there have been some studies on the problem of abnormal detection of cloth image defects:

[0008] For example, Chinese invention patent application number CN202010339971.4 discloses a cloth defect detection method based on One-Class deep support vector description. Under semi-supervised conditions, it takes advantage of the deep convolutional neural network to extract effective deep features of the image. Through training, a hypersphere model that can accurately describe normal sample points is mapped in a high-dimensional space. End-to-end cloth defect detection is achieved. The network parameters and hypersphere radius obtained through training can fully describe the relationship between the test sample and the hypersphere, thereby realizing the identification of defects. It solves the problems of large memory usage, slow detection speed, and limited defect types in previous models.

[0009] For example, Chinese invention patent application number CN202311189949.6 discloses a method for rapid detection of textile defects based on image features. By acquiring a textile grayscale image and performing block processing, all attributed pixel pairs and all original pixel pairs in each reference direction in each sliding window are obtained, and the overall adjustment amount of the pixel pairs of the reset textile grayscale image in each reference direction is calculated based on these data, thereby obtaining the error degree of the reset textile grayscale image in each reference direction. By analyzing the regularity of the texture distribution of the image block, textile defects can be quickly detected. This method has the advantages of being fast, accurate, and efficient.

[0010] For another example, Chinese invention patent application No. CN202311052085.3 discloses a fabric defect detection method based on convolutional neural network and repetitive pattern analysis, including: collecting fabric defect images, performing pixel-level annotation and data enhancement processing on the fabric defect images, establishing a fabric defect image data set, and dividing the data into a training set and a validation set; building a repetitive pattern detection model for repetitive pattern detection; inputting the fabric defect image into the repetitive pattern detection model, obtaining the output feature map and adjusting the size as the input label of the semantic segmentation network for defect detection, extracting the peak value from it and converting it into the target label of the semantic segmentation network for defect detection by unique hot encoding; building and training the semantic segmentation network, and after the training is completed, saving the necessary part of the semantic segmentation network for fabric defect detection to obtain a fabric defect detection model; using the saved fabric defect detection model and the repetitive pattern detection model to perform fabric defect detection, which is suitable for periodic fabric defect detection with high detection accuracy.

[0011] For another example, Chinese invention patent application number CN202211231211.7 discloses a method for identifying defective lines of fabrics for textile production. The method obtains an image of the fabric after sewing and semantically segments the image of the fabric area to be detected. According to the regional features on the fabric, the defective lines are identified. According to the position of the defective lines, the stretching degree of the corresponding lines is adjusted. Based on the digitalization of industry, the present invention uses image processing technology to realize the identification and enhancement of defective lines, so as to better facilitate the identification of whether the fabric contains defects in defect detection.

[0012] The defects of the above invention patent applications are: these methods have limited processing capabilities when facing complex textures and detailed cloth images, especially in the extraction and processing of high-frequency features and fine-grained features, resulting in inaccurate detection of edge areas and subtle defects and low precision. The problems addressed by this patent are more in line with the actual needs of the current industry and have greater application prospects. Summary of the invention

[0013] The object of the present invention is to provide a cloth defect detection method with high-frequency feature constraints and edge entropy minimization.

[0014] The method comprises the following specific steps:

[0015] Step 1. Collect a dataset of cloth training images without defects and anomalies, initialize the hyperparameters, and initialize and fine-tune the model parameters;

[0016] Step 2. Randomly sample an image sample x, obtain vector quantized image features through the feature encoder, generate defect noise through the anomaly generator and add it to the original features of the image, input the features with anomalies into the image reconstruction decoder and the image restoration decoder respectively, and reconstruct the images without anomalies and with anomalies respectively;

[0017] Step 3. Calculate the high-frequency feature constraint loss Supervise the quality of the reconstructed de-anomaly image to ensure the reconstruction quality of detailed texture features;

[0018] Step 4. Calculate the difference image between the reconstructed original input image and the reconstructed de-anomaly image to enhance the features of the abnormal area;

[0019] Step 5. Input the reconstructed original input image, the reconstructed de-anomaly image, and the difference image between the two into the anomaly detector at the same time, and calculate the anomaly detection loss L det ;

[0020] Step 6. Calculate the entropy loss through the edge entropy minimization module Optimize the prediction of edge areas with low certainty to improve the model's confidence in the prediction of edge areas;

[0021] Step 7. Calculate the joint loss And back-propagate to update the parameters;

[0022] Step 8. Repeat steps 2 to 7 until the maximum number of iterations I is reached or the model parameters converge;

[0023] Step 9. During the model usage phase: input the test image into the trained cloth defect anomaly detection model, perform defect detection on the test image, and select the results with a confidence level greater than a certain threshold Φ in the model output anomaly score results as the abnormal area.

[0024] Furthermore, in step 1, data collection and model initialization, the specific steps are:

[0025] Step 101: In an actual cloth textile factory operation environment, use a camera module to capture and photograph different types of flawless cloth, so that each picture can clearly show the appearance of the cloth;

[0026] Step 102, setting the balance parameters α and β and the maximum number of iterations I;

[0027] Step 103: Initialize the model using a fully pre-trained deep feature extractor, image restoration decoder, and feature quantization codebook, and then fine-tune the model parameters using the data of the target data set to minimize the image restoration loss and the difference between the feature space projection F calculated by the deep feature extractor and its codebook vectorized feature Q. The loss function can be expressed as:

[0028]

[0029] In the above formula (1), F represents the feature space projection calculated by the deep feature extractor, Q represents its corresponding codebook vectorized feature, and x rec represents the input image restored by the image restoration decoder, sg[·] indicates stopping gradient calculation, and L2 represents the L2 norm, which is used to measure the distance between two points at corresponding positions of the original image and the generated image. After the first stage of training, the feature encoder, image restoration decoder, and feature quantization codebook will be fixed.

[0030] Furthermore, in step 2, the input image is encoded and decoded and abnormal features are added, and the specific steps are as follows:

[0031] Step 201, feature extraction: The input image x is extracted by a deep feature extractor to obtain features f(x). The deep feature extractor consists of 16 residual blocks. Each residual block contains two 3x3 convolutional layers with a step size of 1 and a 1x1 convolutional layer. Each residual block contains a skip connection, which directly adds the input of the block to the output.

[0032] Step 202, feature quantization: In order to facilitate the generation of abnormal area data that is closer to the normal distribution, the input image is encoded by vector quantization. Vector quantization is a technology that maps a continuous latent space into discrete features. By introducing a discrete codebook in the latent space, the model (no longer marked below) can more effectively learn the high-level abstract representation of the data. By introducing a discrete codebook, vector quantization divides the latent space into a finite number of discrete regions, each of which is represented by a specific codebook vector. Specifically, the feature quantization process of the feature f(x) extracted by the deep feature extractor is expressed as:

[0033]

[0034] In the above formula (2), f(x) represents the features extracted by the deep feature extractor from the input image x, f(x) ij The feature representing the position of the feature map (i, j), e k represents a vector in the codebook, where k is the index of the codebook vector, and the value range is k∈(1…K), where K is the number of features in the codebook, and argmin represents the e that minimizes the function. k The value of , ||·|| represents the Euclidean distance. Finally, each point in the feature space is replaced by the nearest neighbor vector in the codebook;

[0035] Step 203, using two-level vector quantization to improve the quality of the reconstruction result: using low-resolution and high-resolution codebooks to perform two-level encoding on the input image, the quantized feature map of the low-resolution codebook is reduced by 4 times compared with the original input image, while the quantized feature map of the high-resolution codebook is reduced by 8 times. Through this two-level encoding method, the multi-level features of the input image can be better captured, and a higher quality reconstruction result can be produced;

[0036] Step 204, constructing a cloth feature anomaly generation module: in the training stage, the model can only access input images without anomalies, and anomalies need to be generated for the input images. The current noise addition methods are divided into two types: image level and feature level. Among them, the image level noise generation can simulate limited anomalies, while the feature level noise addition is more robust. By introducing noise on specific features of the image, the feature level noise addition can simulate the changes in abnormal features, thereby greatly increasing the diversity of abnormal information. In order to take into account the randomness of the abnormal area while making the model have better resolution ability for anomalies that are difficult to distinguish, the two methods of random noise addition and quantized subspace noise addition are combined when performing anomaly generation: first, random sampling generates a mask Y of the abnormal area, where the area with a value of 1 represents an anomaly, and the random noise addition superimposes the generated random noise on the features of the selected area; the quantized subspace noise addition method is based on the quantized features, randomly selecting features from the feature space of the image data to replace the features of the selected area, while limiting the similarity between the replaced features and the original features without anomalies, and excluding the most similar 5% of vectors;

[0037] Step 205, constructing a cloth image decoder: In the image decoding stage, an image reconstruction decoder and an image restoration decoder are introduced respectively, wherein the image reconstruction decoder aims to restore the abnormal area to the normal appearance of the sample observed during training, while the image restoration decoder directly restores the input features to an image;

[0038] Step 206, for the image reconstruction decoder: the decoder performs feature extraction through two convolutional blocks, where the first convolutional block contains a 3x3 convolutional layer to maintain the number of input channels, followed by an instance normalization layer and a ReLU activation function, followed by another 3x3 convolutional layer to double the number of channels, followed by another normalization layer and a ReLU activation function, and the second convolutional block operates in a similar manner to the first convolutional layer. Each convolutional block is followed by a maximum pooling layer, all using a 2x2 window and a stride of 2 for the next step. Sampling, next, the feature map passes through a 1x1 convolution layer to reduce the number of channels to 64, and then the feature map passes through two upsampling blocks, each of which uses a 4x4 transposed convolution with a step size of 2 to achieve upsampling operations, and a ReLU activation function is connected after each upsampling block. After that, the feature map passes through a 3x3 convolution layer and enters two transposed convolution layers to gradually reduce the number of channels for reconstructing the RGB channels of the original image. After each transposed convolution layer, except for the last layer, a ReLU activation function is connected. Finally, the network outputs the reconstructed image;

[0039] Step 207, for the image restoration decoder: the decoder uses a fixed pre-trained model to achieve direct reconstruction of the input features. The decoder includes a convolution layer for preliminary processing of the potential representation, and extracts deep features through residual stacking. In the residual stacking, each residual block contains two convolution layers with ReLU activation functions, which are upsampled through a deconvolution layer, and then further extracted features through a convolution layer with a ReLU activation function, and then the potential representation is mapped to the space of the original image through another deconvolution layer. The image reconstruction decoder and the image restoration decoder both include operations on input features of two different resolutions. The low-resolution quantized encoded feature map is upsampled and convolved, and then feature extracted through the residual module. The high-resolution feature map is also processed through upsampling and convolution operations. The two feature maps are cascaded together, and the final reconstructed image is generated through transposed convolution.

[0040] Furthermore, in step 3, the high-frequency feature constraint loss is calculated, and the specific steps are as follows:

[0041] Step 301, constructing a high-frequency feature constraint module: In an anomaly detection model based on image reconstruction, the higher the quality of the anomaly-free reconstructed image, the stronger the model's understanding and learning ability of normal samples, which helps to improve the robustness and generalization ability of the model, making it more credible in practical applications. When the model is used for anomaly detection tasks, it can better distinguish normal and abnormal samples. A common way to enhance the reconstruction quality is to use Euclidean distance loss to directly constrain the consistency between the anomaly-free reconstructed image and the original image. However, since the fitting priority of the neural network for frequency information is different throughout the training process, usually from low to high, this will mean that some high-frequency information may not be valued by the network, while the detailed texture information is precisely contained in the high-frequency information;

[0042] In order to narrow the gap between the high-frequency information of the reconstructed de-anomaly image and the original image, a high-frequency feature constraint module is set up, and the original input image and the reconstructed de-anomaly image are converted to the frequency domain using Discrete Fourier Transform (DFT):

[0043]

[0044] In the above formula (3), F(u,v) is the representation of the image in the frequency domain, (u,v) represents the coordinates in the frequency domain, the size of the input image is M×N, x(m,n) represents the pixel value of point (m,n) in the image, e and i are natural constants and imaginary units, respectively, as shown in the following formula (4):

[0045]

[0046] In the above formula (4), cos represents cosine, sin represents sine, (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, x(m, n) represents the pixel value of point (m, n) in the image, e and i are natural constants and imaginary units respectively;

[0047] Step 302, after discrete Fourier transform, the amplitude and phase of the image are two key elements to describe the frequency domain information. The phase describes the spatial position and relative position of the frequency component in the image, determines the starting point of the sine wave in the image, and thus affects the spatial layout of different frequencies in the frequency domain. The amplitude represents the response intensity or energy of the image to a specific frequency. In the centralized amplitude diagram, the low-frequency information is in the middle and the high-frequency information is around. In order to design a loss function that constrains high-frequency information, for the real and imaginary parts of F(u,v) in formula (3), let the real part R(u,v) = a, 6. The imaginary part I(u,v) = b, then F(u,v) is expressed as the following formula (5):

[0048] F(u,v)=R(u,v)+I(u,v)i=a+bi……(5),

[0049] In the above formula (5), (u, v) represents the coordinates in the frequency domain, R(u, v) and a represent the real part of the frequency domain information, I(u, v) and b represent the imaginary part of the frequency domain information, and i is the imaginary unit;

[0050] Step 303: Calculate the amplitude of the image, as shown in equation (6):

[0051]

[0052] In the above formula (6), (u, v) represents the coordinates in the frequency domain, R(u, v) and a represent the real part of the frequency domain information, I(u, v) and b represent the imaginary part of the frequency domain information, and |·| represents the modulus of the complex number;

[0053] Step 304: Calculate the phase of the image according to the following formula (7):

[0054]

[0055] In the above formula (7), (u, v) represents the coordinates in the frequency domain, R(u, v) represents the real part of the frequency domain information, I(u, v) represents the imaginary part of the frequency domain information, arctan is the tangent function, and the distance measurement is performed by mapping each frequency value to a Euclidean vector in a two-dimensional space;

[0056] Step 305: Calculate the spatial frequency value at the input image spectrum coordinate (u, v) according to the following formula (8):

[0057] F in (u,v)=ain +b in i……(8),

[0058] In the above formula (8), a in represents the real part of the input image spectrum, b in Represents the imaginary part of the input image spectrum, where i is the imaginary unit;

[0059] Step 306: Calculate the spatial frequency value at the frequency spectrum coordinate (u, v) of the abnormality-removed reconstructed image according to the following formula (9):

[0060] F gen (u,v)=a gen +b gen i……(9),

[0061] In the above formula (9), a gen represents the real part of the spectrum of the image after de-anomalization and reconstruction, b gen represents the imaginary part of the spectrum of the image reconstructed after abnormality removal, i is an imaginary unit;

[0062] Step 307: Calculate the global distance of the frequency using the square of the Euclidean distance, as shown in the following formula (10):

[0063]

[0064] In the above formula (10), and Indicates that from F in (u,v) and F gen The two vectors mapped by (u, v) are based on the definition of amplitude and phase. and Corresponding to the amplitude, the angle θ in and θ gen Corresponding to the phase, ||·|| represents the Euclidean distance, and |·| represents the modulus of the complex number;

[0065] Step 308: Calculate the frequency distance between the original input image and the de-anomalized reconstructed image, using the average value of the global distance, as shown in the following formula (11):

[0066]

[0067] In the above formula (11), F in represents the input image spectrum, F gen represents the spectrum of the image reconstructed by removing abnormalities, (u,v) represents the coordinates in the frequency domain, the size of the input image is M×N, and |·| represents the modulus of the complex number;

[0068] Step 309: In order to make the model focus more on learning key fine-grained features, filter out low-frequency information, reduce the model's attention to overall structural information, and capture subtle textures and detail features in the image, a mask M is introduced here. r , assuming that the center point of the image is (0, 0), then M r The size of is M×N, the value of the central area rM×rN is 0, and the value of the rest of the area is 1, where r∈(0,1). The mask function is calculated as follows:

[0069]

[0070] In the above formula (12), (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, and r is the ratio of the area size of the central region. is an indicator function used to determine whether the frequency domain coordinates (u, v) are outside the specified central area. If they are outside the area, the mask function M r (u,v) takes the value of 1; if it is within the area, it takes the value of 0;

[0071] Step 310: Calculate the high-frequency feature constraint loss according to the following formula (13):

[0072]

[0073] In the above formula (13), (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, r is the ratio of the area size of the central region, and F in represents the input image spectrum, F gen Represents the spectrum of the image after removing abnormalities and reconstructing it.

[0074] Furthermore, in step 4, a defect feature enhancement module is constructed, and the specific steps are as follows:

[0075] Step 401, constructing a defect feature enhancement module to receive two inputs, namely, a reconstructed original input image and a reconstructed de-anomaly image;

[0076] Step 402, calculating a difference image between the reconstructed original input image and the reconstructed de-anomaly image, wherein the difference image is obtained by directly subtracting the original input image and the de-anomaly image, so as to highlight the difference area between the images and enhance the features of the abnormal area;

[0077] Step 403: splice the reconstructed original input image, the reconstructed anomaly-removed image, and the difference image between the two, and input them into a cloth defect anomaly detection module.

[0078] Furthermore, in step 5, a cloth defect anomaly detection module is constructed, and the specific steps are as follows:

[0079] Step 501, construct a cloth defect anomaly detection module: The function of the anomaly detection module is to generate an anomaly prediction for each pixel point, which is presented in the form of an anomaly score, indicating the degree of difference between the input image and the normal mode. The anomaly detection module accepts three input images, including the reconstructed original input image, the reconstructed anomaly-free image, and the difference image between the above two, and predicts the anomaly score through a deep defect detection network. The deep defect detection network includes an encoder for capturing hierarchical features and a decoder for reconstructing high-resolution anomaly predictions, wherein: the encoder part is composed of four convolution blocks, each convolution block includes two 3x3 convolution layers for feature extraction, the output channel number of the first convolution layer is consistent with the input, and the output channel number of the second convolution layer is the first Twice, each convolution layer is followed by an instance normalization layer and a ReLU activation function. Each pair of convolution layers is followed by a 2x2 maximum pooling layer for downsampling, which gradually reduces the spatial size of the feature map while increasing the number of feature channels. The decoder part consists of upsampling and convolution blocks, which are used to gradually restore the spatial resolution of the image. Each upsampling step uses bilinear interpolation and is followed by a 3x3 convolution layer to adjust the number of channels, followed by a ReLU activation function. Each upsampling step is followed by a convolution block, which includes two 3x3 convolution layers for further feature refinement. In each upsampling step, the decoder also performs a jump connection with the feature map of the corresponding layer of the encoder. Finally, the output of the decoder is mapped to the required number of output channels through a 1x1 convolution layer.

[0080] Step 502, calculate the defect detection loss: train by minimizing the anomaly detection loss function between the prediction score P and the anomaly region mask Y. The loss function is expressed as the following formula (14):

[0081]

[0082] In the above formula (14), Y i is the true label of the i-th pixel (0 for normal or 1 for abnormal), P i is the predicted value of the i-th pixel, is a balancing factor used to balance the ratio of positive and negative samples, and γ is an adjustment factor used to adjust the weight of difficult and easy samples.

[0083] Furthermore, in step 6, an edge entropy minimization module is constructed, and the specific steps are as follows:

[0084] Step 601, when performing pixel-level abnormal region prediction, the model will generate low entropy predictions in the central part close to the abnormal region, and high entropy predictions in the edge part of the abnormal region. Specifically, the abnormal features are usually more significant in the central part of the abnormal region, and the model is more likely to learn the common features of these central regions, thereby generating confident predictions for the central region. When the model faces the edge part of the abnormal region, due to the difference and complexity of the data distribution in this part or insufficient abnormal information, the model cannot accurately capture the detailed information of the edge part, resulting in insufficiently confident high entropy predictions in these areas. This shows that only using the supervision mode of the above step 502 is still not enough to enable the model to generate high-certainty predictions for the abnormal edge region. One way to improve the detection performance of the abnormal detection model in the edge part is to encourage the model to generate high-confidence (low entropy) predictions in the target prediction stage. Here, an edge entropy minimization module is set, and the design uses entropy loss to directly punish low-confidence predictions;

[0085] Specifically, the edge entropy minimization module does not consider using the generated abnormal area mask Y to improve the edge prediction confidence, but considers a more direct constraint method, that is, to achieve it by minimizing the predicted entropy. According to the Shannon entropy principle, for the input image x, its entropy map is composed of independent pixel-level entropies, and the entropy value of each pixel point is as follows (15):

[0086]

[0087] In the above formula (15), Represents the model's abnormal prediction value for the image (m,n) position;

[0088] Step 602: In order to improve the confidence of the anomaly detection model in the edge uncertainty area, a reweighting mechanism is introduced. Since the predicted values ​​in the edge uncertainty area are all close to 0.5, the distance between the predicted absolute value and 0.5 is calculated, and a greater loss supervision is imposed on the pixels closer to 0.5. For the (m, n) position of the image, a distance weight is introduced. The following formula (16):

[0089]

[0090] In the above formula (16), λ is a hyperparameter that controls the weight decay speed, and its value range is between 0 and 1. The value of λ directly affects the decay speed of the distance weight. The larger the λ is, the faster the decay speed is, and the weight of the prediction value with a longer distance decreases faster; while the smaller the λ is, the slower the decay speed is, and the weight of the prediction value with a longer distance decreases relatively slowly.

[0091] Step 603: Entropy loss is defined as the sum of all pixel-level entropies, as shown in equation (17):

[0092]

[0093] In the above formula (17), Represents the entropy value at the pixel point (m,n), Represents the distance weight at the pixel point (m,n).

[0094] Furthermore, in step 7, the joint training module includes the following specific steps:

[0095] Step 701: Calculate the joint loss according to the following formula (18): To update the parameters of the image reconstruction decoder and anomaly detector:

[0096]

[0097] In the above formula (18), is the anomaly detection loss, is the consistency loss between the abnormal reconstructed image and the original image, is the high-frequency feature constraint loss, is the entropy loss, α and β are and The weight factor of .

[0098] Furthermore, step 9, the actual use of the model, the specific steps are:

[0099] Step 901: For a defective input image, it first passes through a feature encoder to obtain features with abnormal areas;

[0100] Step 902: Send the features of the abnormal area to the image reconstruction decoder and the image restoration decoder respectively, and reconstruct the image without the abnormality and the image with the abnormality respectively;

[0101] Step 903, stitching the generated anomaly image / abnormal image and the difference image between the two, and inputting them into an anomaly detector at the same time;

[0102] Step 904: predict the abnormal possibility of each pixel point, and the score of the pixel point in the defective area of ​​the final prediction result will be closer to 1, and the score of the pixel point in the non-defective area will be closer to 0;

[0103] Step 905: For an input image without defects, after the same reasoning process, the defect anomaly score of the entire image area will be relatively close to 0.

[0104] Compared with the existing similar methods in the art, the method of the present invention has the following advantages:

[0105] 1. Using the method described in the present invention, cloth defect anomaly detection is more accurate. By introducing a high-frequency feature constraint module and an edge entropy minimization module, the accuracy of cloth defect anomaly detection is significantly improved; the high-frequency feature constraint module uses discrete Fourier transform to convert the real image and the reconstructed image into frequency domain representation, and ensures the consistency of the reconstructed image and the original image in detail texture by reducing the difference between high-frequency information; the edge entropy minimization module punishes low-confidence predictions in defect abnormal areas by minimizing entropy loss, and enhances the model's prediction certainty for defect abnormal edge areas. Through the combination of the high-frequency feature constraint module and the edge entropy minimization module, the method described in the present invention can more accurately detect subtle defects and abnormal areas on the cloth surface, thereby improving the accuracy of detection.

[0106] 2. Using the method described in the present invention, the reconstruction of the cloth defect area is high in precision. Compared with the existing cloth defect anomaly detection method, the method described in the present invention supervises the quality of the reconstructed anomaly-free image through a high-frequency feature constraint module to ensure the reconstruction quality of the detail texture features. The established module constrains the high-frequency features in the frequency domain to ensure the consistency of the reconstructed image in the detail texture, thereby improving the precision of the reconstructed image. Regardless of whether it is an anomaly-free image or an image with anomalies, the reconstruction result can retain more detail information, so that the precision of the reconstructed image is significantly improved.

[0107] 3. Using the method described in the present invention, the detection accuracy of the defect edge area is higher. Compared with the existing cloth defect anomaly detection method, the edge entropy minimization module of the method described in the present invention optimizes the predictions with low certainty in the anomaly detection stage to improve the model's confidence in the edge area prediction. By minimizing the entropy loss, the module penalizes the low-confidence predictions in the defect anomaly area, thereby enhancing the model's detection accuracy of the defect anomaly edge area. In this way, the model can not only detect the defect area, but also accurately identify the edges of the defects, so that the overall detection accuracy is higher, especially when processing complex industrial cloth images, the effect is particularly significant.

[0108] 4. Using the method described in the present invention, cloth defect anomaly detection training samples do not need to be labeled. During the model training process, only defect-free cloth image data is required, and there is no need to label the training samples. This advantage significantly reduces the cost and time of data preparation, and avoids the subjective errors caused by manual labeling. Through unsupervised learning, the model can automatically learn and detect defect abnormal areas in cloth images, making the entire detection process more efficient and automated, and meeting the needs of large-scale industrial production.

[0109] 5. When processing cloth image defect anomaly detection tasks in complex industrial environments, the method described in the present invention can significantly enhance the fine-grained features of the reconstructed image and the defect anomaly detection effect in the edge area, improve the fineness of the reconstructed image and the accuracy of defect anomaly detection, and provide a new solution for cloth image defect anomaly detection, which has broad application prospects and important practical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] Figure 1 The present invention is a flowchart of the method.

[0111] Figure 2 This is a framework diagram of the training phase of the method described in the present invention.

[0112] FIG3 is a framework diagram of the test reasoning phase of the method of the present invention, wherein Figure 3A This is the framework diagram of the reasoning stage of the defective cloth image test. Figure 3B This is the framework diagram for the inference stage of the flawless cloth image test. DETAILED DESCRIPTION

[0113] In order to make the purpose, technical solution and advantages of the method of the present invention more clear, the method of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0114] The method of the present invention comprises the following steps:

[0115] like Figure 1 , Figure 2 As shown, step 1, data collection and model initialization, the specific steps are:

[0116] Step 101: In an actual cloth textile factory operation environment, use a camera module to capture and photograph different types of flawless cloth, so that each picture can clearly show the appearance of the cloth;

[0117] Step 102, setting the balance parameters α and β and the maximum number of iterations I;

[0118] Step 103: Initialize the model using a fully pre-trained deep feature extractor, image restoration decoder, and feature quantization codebook, and then fine-tune the model parameters using the data of the target data set to minimize the image restoration loss and the difference between the feature space projection F calculated by the deep feature extractor and its codebook vectorized feature Q. The loss function is expressed as:

[0119]

[0120] In the above formula (1), F represents the feature space projection calculated by the deep feature extractor, Q represents its corresponding codebook vectorized feature, and x recrepresents the input image restored by the image restoration decoder, sg[·] indicates stopping gradient calculation, and L2 represents the L2 norm, which is used to measure the distance between two points at corresponding positions of the original image and the generated image. After the first stage of training, the feature encoder, image restoration decoder, and feature quantization codebook will be fixed;

[0121] Step 2: Encode and decode the input image and add abnormal features, such as Figure 2 As shown, the specific steps are:

[0122] Step 201, feature extraction: The input image x is extracted by a deep feature extractor to obtain features f(x). The deep feature extractor extracts a total of 16 residual blocks, each of which contains two 3x3 convolutional layers with a step size of 1 and a 1x1 convolutional layer. Each residual block contains a skip connection, which directly adds the input of the block to the output;

[0123] Step 202, feature quantization: In order to facilitate the generation of abnormal area data that is closer to the normal distribution, the input image is encoded by vector quantization. Vector quantization is a technology that maps a continuous latent space into discrete features. By introducing a discrete codebook in the latent space, the model can more effectively learn the high-level abstract representation of the data. By introducing a discrete codebook, vector quantization divides the latent space into a finite number of discrete regions, each of which is represented by a specific codebook vector. Specifically, the feature quantization process of the feature f(x) extracted by the deep feature extractor is expressed as:

[0124]

[0125] In the above formula (2), f(x) represents the features extracted by the deep feature extractor from the input image x, f(x) ij The feature representing the position of the feature map (i, j), e k represents a vector in the codebook, where k is the index of the codebook vector, and the value range is k∈(1…K), where K is the number of features in the codebook, and argmin represents the e that minimizes the function. k The value of , ||·|| represents the Euclidean distance. Finally, each point in the feature space is replaced by the nearest neighbor vector in the codebook;

[0126] Step 203, using two-level vector quantization to improve the quality of the reconstruction result: using low-resolution and high-resolution codebooks to perform two-level encoding on the input image, the quantized feature map of the low-resolution codebook is reduced by 4 times compared with the original input image, while the quantized feature map of the high-resolution codebook is reduced by 8 times. Through this two-level encoding method, the multi-level features of the input image can be better captured, and a higher quality reconstruction result can be produced;

[0127] Step 204, constructing a cloth feature anomaly generation module: in the training stage, the model can only access input images without anomalies, and it is necessary to generate anomalies for the input images. The current noise addition methods are divided into two types: image level and feature level. Among them, the image level noise generation can simulate limited anomalies, while the feature level noise addition is more robust. By introducing noise on specific features of the image, the feature level noise addition can simulate the changes in abnormal features, thereby increasing the diversity of abnormal information. In order to give the model better resolution for anomalies that are difficult to distinguish while taking into account the randomness of the abnormal area, the two methods of random noise addition and random noise addition in the quantized subspace are combined when performing anomaly generation: first, random sampling generates a mask Y of the abnormal area, where the area with a value of 1 represents an anomaly, and the random noise addition superimposes the generated random noise on the features of the selected area; the quantized subspace noise addition method is based on the quantized features, randomly selecting features from the feature space of the image data to replace the features of the selected area, while limiting the similarity between the replaced features and the original features without anomalies, and excluding the most similar 5% of vectors;

[0128] Step 205, constructing a cloth image decoder: In the image decoding stage, an image reconstruction decoder and an image restoration decoder are introduced respectively, wherein the image reconstruction decoder aims to restore the abnormal area to the normal appearance of the sample observed during training, while the image restoration decoder directly restores the input features to an image;

[0129] Step 206, for the image reconstruction decoder: the decoder performs feature extraction through two convolutional blocks, where the first convolutional block contains a 3x3 convolutional layer to maintain the number of input channels, followed by an instance normalization layer and a ReLU activation function, followed by another 3x3 convolutional layer to double the number of channels, followed by another normalization layer and a ReLU activation function, and the second convolutional block operates in a similar manner to the first convolutional layer. Each convolutional block is followed by a maximum pooling layer, all using a 2x2 window and a stride of 2 for the next step. Sampling, next, the feature map passes through a 1x1 convolution layer to reduce the number of channels to 64, and then the feature map passes through two upsampling blocks, each of which uses a 4x4 transposed convolution with a step size of 2 to achieve upsampling operations, and a ReLU activation function is connected after each upsampling block. After that, the feature map passes through a 3x3 convolution layer and enters two transposed convolution layers to gradually reduce the number of channels for reconstructing the RGB channels of the original image. After each transposed convolution layer, except for the last layer, a ReLU activation function is connected. Finally, the network outputs the reconstructed image;

[0130] Step 207, for the image restoration decoder: the decoder uses a fixed pre-trained model to directly reconstruct the input features. The decoder includes a convolution layer for preliminary processing of the potential representation, and extracts deep features through residual stacking. In the residual stacking, each residual block includes two convolution layers with ReLU activation functions, which are upsampled through a deconvolution layer, and further extracted features through a convolution layer with a ReLU activation function, and then the potential representation is mapped to the space of the original image through another deconvolution layer. The image reconstruction decoder and the image restoration decoder both include operations on input features of two different resolutions. The low-resolution quantized encoded feature map is upsampled and convolved, and then feature extracted through the residual module. The high-resolution feature map is also processed through upsampling and convolution operations. The two feature maps are cascaded together, and the final reconstructed image is generated through transposed convolution.

[0131] Step 3: Calculate the high-frequency feature constraint loss. The specific steps are:

[0132] Step 301, constructing a high-frequency feature constraint module: In an anomaly detection model based on image reconstruction, the higher the quality of the anomaly-free reconstructed image, the stronger the model's understanding and learning ability of normal samples, which helps to improve the robustness and generalization ability of the model, making it more credible in practical applications. When the model is used for anomaly detection tasks, it can distinguish between normal and abnormal samples. The way to enhance the reconstruction quality is to use the Euclidean distance loss to directly constrain the consistency of the anomaly-free reconstructed image with the original image. Since the fitting priority of the neural network for frequency information is different throughout the training process, usually from low to high, this will mean that some high-frequency information may not be valued by the network, while the detailed texture information is precisely contained in the high-frequency information;

[0133] In order to narrow the gap between the high-frequency information of the reconstructed de-anomaly image and the original image, a high-frequency feature constraint module is set up, and the original input image and the reconstructed de-anomaly image are converted to the frequency domain using Discrete Fourier Transform (DFT):

[0134]

[0135] In the above formula (3), F(u,v) is the representation of the image in the frequency domain, (u,v) represents the coordinates in the frequency domain, the size of the input image is M×N, x(m,n) represents the pixel value of point (m,n) in the image, e and i are natural constants and imaginary units, respectively, as shown in the following formula (4):

[0136]

[0137] In the above formula (4), cos represents cosine, sin represents sine, (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, x(m, n) represents the pixel value of point (m, n) in the image, e and i are natural constants and imaginary units respectively;

[0138] Step 302, after discrete Fourier transform, the amplitude and phase of the image are two key elements to describe the frequency domain information. The phase describes the spatial position and relative position of the frequency component in the image, determines the starting point of the sine wave in the image, and thus affects the spatial layout of different frequencies in the frequency domain. The amplitude represents the response intensity or energy of the image to a specific frequency. In the centralized amplitude diagram, the low-frequency information is in the middle and the high-frequency information is around. In order to design a loss function that constrains high-frequency information, for the real and imaginary parts of F(u,v) in formula (3), let the real part R(u,v) = a, 6. The imaginary part I(u,v) = b, then F(u,v) is expressed as the following formula (5):

[0139] F(u,v)=R(u,v)+I(u,v)i=a+bi……(5),

[0140] In the above formula (5), (u, v) represents the coordinates in the frequency domain, R(u, v) and a represent the real part of the frequency domain information, I(u, v) and b represent the imaginary part of the frequency domain information, and i is the imaginary unit;

[0141] Step 303: Calculate the amplitude of the image, as shown in equation (6):

[0142]

[0143] In the above formula (6), (u, v) represents the coordinates in the frequency domain, R(u, v) and a represent the real part of the frequency domain information, I(u, v) and b represent the imaginary part of the frequency domain information, and |·| represents the modulus of the complex number;

[0144] Step 304: Calculate the phase of the image according to the following formula (7):

[0145]

[0146] In the above formula (7), (u, v) represents the coordinates in the frequency domain, R(u, v) represents the real part of the frequency domain information, I(u, v) represents the imaginary part of the frequency domain information, arctan is the tangent function, and the distance measurement is performed by mapping each frequency value to a Euclidean vector in a two-dimensional space;

[0147] Step 305: Calculate the spatial frequency value at the input image spectrum coordinate (u, v) according to the following formula (8):

[0148] F in (u,v)=ain +b in i……(8),

[0149] In the above formula (8), a in represents the real part of the input image spectrum, b in Represents the imaginary part of the input image spectrum, where i is the imaginary unit;

[0150] Step 306: Calculate the spatial frequency value at the frequency spectrum coordinate (u, v) of the abnormality-removed reconstructed image according to the following formula (9):

[0151] F gen (u,v)=a gen +b gen i……(9),

[0152] In the above formula (9), a gen represents the real part of the spectrum of the image after de-anomalization and reconstruction, b gen represents the imaginary part of the spectrum of the image reconstructed after abnormality removal, i is an imaginary unit;

[0153] Step 307: Calculate the global distance of the frequency using the square of the Euclidean distance, as shown in the following formula (10):

[0154]

[0155] In the above formula (10), and Indicates that from F in (u,v) and F gen The two vectors mapped by (u, v) are based on the definition of amplitude and phase. and Corresponding to the amplitude, the angle θ in and θ gen Corresponding to the phase, ||·|| represents the Euclidean distance, and |·| represents the modulus of the complex number;

[0156] Step 308: Calculate the frequency distance between the original input image and the de-anomalized reconstructed image, using the average value of the global distance, as shown in the following formula (11):

[0157]

[0158] In the above formula (11), F in represents the input image spectrum, F gen represents the spectrum of the image reconstructed by removing abnormalities, (u,v) represents the coordinates in the frequency domain, the size of the input image is M×N, and |·| represents the modulus of the complex number;

[0159] Step 309: In order to make the model focus more on learning key fine-grained features, filter out low-frequency information, reduce the model's attention to overall structural information, and capture subtle textures and detail features in the image, a mask M is introduced here. r , assuming that the center point of the image is (0, 0), then M r The size of is M×N, the value of the central area rM×rN is 0, and the value of the rest of the area is 1, where r∈(0,1). The mask function is calculated as follows:

[0160]

[0161] In the above formula (12), (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, and r is the ratio of the area size of the central region. is an indicator function used to determine whether the frequency domain coordinates (u, v) are outside the specified central area. If they are outside the area, the mask function M r (u,v) takes the value of 1; if it is within the area, it takes the value of 0;

[0162] Step 310: Calculate the high-frequency feature constraint loss according to the following formula (13):

[0163]

[0164] In the above formula (13), (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, r is the ratio of the area size of the central region, and F in represents the input image spectrum, F gen represents the spectrum of the image reconstructed by removing abnormalities;

[0165] Step 4: Build a defect feature enhancement module. The specific steps are as follows:

[0166] Step 401, constructing a defect feature enhancement module to receive two inputs, namely, a reconstructed original input image and a reconstructed de-anomaly image;

[0167] Step 402, calculating a difference image between the reconstructed original input image and the reconstructed de-anomaly image, wherein the difference image is obtained by directly subtracting the original input image and the de-anomaly image, so as to highlight the difference area between the images and enhance the features of the abnormal area;

[0168] Step 403: splice the reconstructed original input image, the reconstructed abnormality-removed image, and the difference image between the two, and input them into a cloth defect abnormality detection module;

[0169] Step 5: Build a cloth defect anomaly detection module. The specific steps are as follows:

[0170] Step 501, construct a cloth defect anomaly detection module: The function of the anomaly detection module is to generate an anomaly prediction for each pixel point, which is presented in the form of an anomaly score, indicating the degree of difference between the input image and the normal mode. The anomaly detection module accepts three input images, including the reconstructed original input image, the reconstructed anomaly-free image, and the difference image between the above two, and predicts the anomaly score through a deep defect detection network. The deep defect detection network includes an encoder for capturing hierarchical features and a decoder for reconstructing high-resolution anomaly predictions; wherein: the encoder part is composed of four convolution blocks, each convolution block includes two 3x3 convolution layers for feature extraction, the output channel number of the first convolution layer is consistent with the input, and the output channel number of the second convolution layer is the first Twice, each convolution layer is followed by an instance normalization layer and a ReLU activation function. Each pair of convolution layers is followed by a 2x2 maximum pooling layer for downsampling, which gradually reduces the spatial size of the feature map while increasing the number of feature channels. The decoder part consists of upsampling and convolution blocks, which are used to gradually restore the spatial resolution of the image. Each upsampling step uses bilinear interpolation and is followed by a 3x3 convolution layer to adjust the number of channels, followed by a ReLU activation function. Each upsampling step is followed by a convolution block, which includes two 3x3 convolution layers for further feature refinement. In each upsampling step, the decoder also performs a jump connection with the feature map of the corresponding layer of the encoder. Finally, the output of the decoder is mapped to the required number of output channels through a 1x1 convolution layer.

[0171] Step 502, calculate the defect detection loss: train by minimizing the anomaly detection loss function between the prediction score P and the anomaly region mask Y. The loss function is expressed as the following formula (14):

[0172]

[0173] In the above formula (14), Y i is the true label of the i-th pixel (0 for normal or 1 for abnormal), P i is the predicted value of the i-th pixel, θ is the balance factor used to balance the ratio of positive and negative samples, and γ is the adjustment factor used to adjust the weight of difficult and easy samples;

[0174] Step 6: Construct the edge entropy minimization module. The specific steps are:

[0175] Step 601, when performing pixel-level abnormal region prediction, the model will generate a low entropy prediction near the center of the abnormal region, and a high entropy prediction at the edge of the abnormal region. Specifically, the abnormal features are usually more significant in the center of the abnormal region, and the model is more likely to learn the common features of these central regions, thereby generating a confident prediction of the central region. When the model faces the edge of the abnormal region, due to the difference and complexity of the data distribution of this part or insufficient abnormal information, the model cannot accurately capture the detailed information of the edge, resulting in an insufficiently confident high entropy prediction in these areas. Only using the supervision mode of the above step 502 is still not enough to enable the model to generate a high-certainty prediction of the abnormal edge region. One way to improve the detection performance of the abnormal detection model in the edge part is to encourage the model to generate a high-certainty (low-entropy) prediction in the target prediction stage. Here, an edge entropy minimization module is set to directly punish low-certainty predictions using entropy loss;

[0176] The edge entropy minimization module does not consider using the generated abnormal area mask Y to improve the edge prediction confidence, but considers a more direct constraint method, that is, to achieve it by minimizing the predicted entropy. According to the Shannon entropy principle, for the input image x, its entropy map is composed of independent pixel-level entropies, and the entropy value of each pixel point is as follows (15):

[0177]

[0178] In the above formula (15), Represents the model's abnormal prediction value for the image (m,n) position;

[0179] Step 602: In order to improve the confidence of the anomaly detection model in the edge uncertainty area, a reweighting mechanism is introduced. Since the predicted values ​​in the edge uncertainty area are all close to 0.5, the distance between the predicted absolute value and 0.5 is calculated, and a greater loss supervision is imposed on the pixels closer to 0.5. For the (m, n) position of the image, a distance weight is introduced. The following formula (16):

[0180]

[0181] In the above formula (16), λ is a hyperparameter that controls the weight decay speed, and its value range is between 0 and 1. The value of λ directly affects the decay speed of the distance weight. The larger the λ is, the faster the decay speed is, and the weight of the prediction value with a longer distance decreases faster; while the smaller the λ is, the slower the decay speed is, and the weight of the prediction value with a longer distance decreases relatively slowly.

[0182] Step 603: Entropy loss is defined as the sum of all pixel-level entropies, as shown in equation (17):

[0183]

[0184] In the above formula (17), Represents the entropy value at the pixel point (m,n), Represents the distance weight at the pixel point (m,n);

[0185] Step 7, joint training module, the specific steps are:

[0186] Step 701: Calculate the joint loss according to the following formula (18): To update the parameters of the image reconstruction decoder and anomaly detector:

[0187]

[0188] In the above formula (18), is the anomaly detection loss, is the consistency loss between the abnormal reconstructed image and the original image, is the high-frequency feature constraint loss, is the entropy loss, α and β are and The weight factor of

[0189] Step 8, repeat steps 2 to 7 until the maximum number of iterations I is reached or the model parameters converge;

[0190] Step 9, actual use of the model, the specific steps are:

[0191] Step 901: For an input image with defects, Figure 3A As shown, it first passes through the feature encoder to obtain features with abnormal areas;

[0192] Step 902: Send the features of the abnormal area to the image reconstruction decoder and the image restoration decoder respectively, and reconstruct the image without the abnormality and the image with the abnormality respectively;

[0193] Step 903, stitching the generated anomaly image / abnormal image and the difference image between the two, and inputting them into an anomaly detector at the same time;

[0194] Step 904: predict the abnormal possibility of each pixel point, and the score of the pixel point in the defective area of ​​the final prediction result will be closer to 1, and the score of the pixel point in the non-defective area will be closer to 0;

[0195] Step 905: For an input image without defects, Figure 3B As shown in Figure 1, after the same reasoning process, the defect anomaly score of the entire image area will be closer to 0.

[0196] The random generation method of abnormal areas in the method of the present invention is as follows:

[0197] Different area shapes that can be generated on an image by the random generation method of abnormal areas include rectangles, circles, triangles, ellipses, and stars. Within a reasonable area, the coordinates of the center point of the abnormal area are randomly determined, and the parameters required to generate the shape are randomly determined; taking the random generation of an elliptical area as an example, first the width and height of the target image are obtained, and then the coordinates of the center point of the ellipse (cx, cy) are randomly generated, where cx and cy are within the range of the image width and height, respectively, and then the major and minor axis lengths a and b of the ellipse are randomly selected to ensure that they are smaller than the corresponding size of the image, and finally the rotation angle of the ellipse is randomly selected, usually between 0 and 180 degrees. After all the parameters of the ellipse are determined, the corresponding elliptical area can be determined on the image. In actual training, the method of the present invention randomly selects one of the two noise generation methods of random noise addition and quantized subspace noise addition with a probability of 0.5 to generate noise at the feature level.

[0198] Parameter settings for model training of the method described in the present invention:

[0199] In the model initialization and model fine-tuning phase of step 1, the deep feature extractor, image restoration decoder, and feature quantization codebook were trained on the ImageNet dataset for image reconstruction. A total of 200,000 iterations were performed, with a batch size of 32 and a learning rate of 2×10 -4 , the codebook consists of 4096 vectors of length 128;

[0200] In the repeated training phase of step 8 of the Summary of the Invention, the method of the present invention performs 5×10 4 The training was repeated for 10 times, with the batch size set to 8 and the learning rate set to 2×10. -4 In addition, when the training reached 3.5×10 4 At the iteration, the learning rate is reduced to 2×10 -5 At the same time, the value of r in formula (13) is 0.09, and the value of r in formula (14) is The value is 0.75, the value of γ in formula (14) is 2, the value of λ in formula (16) is 0.9, the value of α in formula (18) is 20, and the value of β is 10;

[0201] In order to demonstrate the performance of the method of the present invention in the task of cloth image defect detection, the model performance was tested on actual cloth image data using conventional indicators (Area Under the Receiver Operating Characteristic, AUROC) to test the model performance. The image-level AUROC is used to evaluate the anomaly detection capability of the model. This indicator measures the ability of the model to distinguish between normal images and images with defects at the entire image level, denoted as AUROC. img ; AUROC at the pixel level is used to evaluate the model's ability to locate anomalies. This indicator measures the model's ability to distinguish between normal pixels and defective pixels at the single pixel level, denoted as AUROC pix , to jointly evaluate the anomaly localization performance of the model, specifically:

[0202] True Positive (TP) refers to the number of samples that the model correctly predicts as positive. For example, in defect detection, if the model detects a defect and there is indeed a defect, this will be counted as TP.

[0203] False Negative (FN) refers to the number of positive samples that the model incorrectly predicts as negative. In the context of defect detection, if the model does not detect a defect that actually exists, this will be counted as FN.

[0204] False Positive (FP) refers to the number of negative samples that the model incorrectly predicts as positive. In defect detection, if the model mistakenly identifies a flawless part as defective, this will be counted as FP.

[0205] True Negative (TN) refers to the number of samples that the model correctly predicts as negative. That is, when the model correctly identifies the part of the image without defects, it will be counted in TN.

[0206] The true positive rate TPR and the false positive rate FPR are shown as follows:

[0207]

[0208] The precision Precision and recall Recall are shown as follows:

[0209]

[0210] Recall = TPR,

[0211] AUROC img The calculation method is:

[0212]

[0213] AUROC pix The calculation method is:

[0214]

[0215] Where N is the total number of pixels.

[0216] From the cloth defect detection results of the method of the present invention in an actual production environment, it can be seen that the method of the present invention can accurately identify the edges of defects when processing industrial cloth images with complex textures and rich details, and the overall detection accuracy is high; in the reconstruction process, the consistency of the reconstructed image with the original image in terms of detail texture can also be improved, thereby ensuring the fineness of the reconstructed image. The method of the present invention achieves a comprehensive improvement in the accuracy of cloth defect anomaly detection, reconstruction fineness and edge area detection accuracy through the synergy between various modules, thereby providing an efficient and reliable technical solution for industrial cloth quality control.

[0217] The present invention is not limited by the above embodiments. The above embodiments and descriptions are only for explaining the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, which are within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the appended claims.

Claims

1. A cloth defect detection method with high-frequency feature constraints and edge entropy minimization, comprising: Step 1. Collect a dataset of cloth training images without defects and anomalies, initialize the hyperparameters, and initialize and fine-tune the model parameters; Step 2. Randomly sample an image sample x, obtain vector quantized image features through the feature encoder, generate defect noise through the anomaly generator and add it to the original features of the image, input the features with anomalies into the image reconstruction decoder and the image restoration decoder respectively, and reconstruct the images without anomalies and with anomalies respectively; Step 3. Calculate the high-frequency feature constraint loss Supervise the quality of the reconstructed de-anomaly image to ensure the reconstruction quality of detailed texture features; Step 4. Calculate the difference image between the reconstructed original input image and the reconstructed de-anomaly image to enhance the features of the abnormal area; Step 5. Input the reconstructed original input image, the reconstructed de-anomaly image, and the difference image between the two into the anomaly detector at the same time, and calculate the anomaly detection loss L det ; Step 6. Calculate the entropy loss through the edge entropy minimization module Optimize the prediction of edge areas with low certainty to improve the model's confidence in the prediction of edge areas; Step 7. Calculate the joint loss And back-propagate to update the parameters; Step 8. Repeat steps 2 to 7 until the maximum number of iterations I is reached or the model parameters converge; Step 9. During the model usage phase, the test image is input into the trained cloth defect anomaly detection model, defect detection is performed on the test image, and the results with a confidence level greater than a certain threshold Φ in the model output anomaly score result are selected as the abnormal area.

2. According to the cloth defect detection method with high-frequency feature constraints and edge entropy minimization as claimed in claim 1, a cloth training image dataset without defect abnormalities is collected in step 1, and the hyperparameters are initialized, and the model parameters are initialized and fine-tuned, and the specific steps are: Step 101: In an actual cloth textile factory operation environment, use a camera module to capture and photograph different types of flawless cloth, so that each picture can clearly show the appearance of the cloth; Step 102, setting the balance parameters α and β and the maximum number of iterations I; Step 103: Initialize the model using a fully pre-trained deep feature extractor, image restoration decoder, and feature quantization codebook, and then fine-tune the model parameters using the data of the target data set to minimize the image restoration loss and the difference between the feature space projection F calculated by the deep feature extractor and its codebook vectorized feature Q. The loss function can be expressed as: In the above formula (1), F represents the feature space projection calculated by the deep feature extractor, Q represents its corresponding codebook vectorized feature, and x rec represents the input image restored by the image restoration decoder, sg[·] indicates stopping gradient calculation, and L2 represents the L2 norm, which is used to measure the distance between two points at corresponding positions of the original image and the generated image. After the first stage of training, the feature encoder, image restoration decoder, and feature quantization codebook will be fixed.

3. According to the method for detecting cloth defects with high-frequency feature constraints and minimization of edge entropy as claimed in claim 1, randomly sampling an image sample x in step 2, obtaining vector quantized image features through a feature encoder, generating defect noise through an anomaly generator and adding it to the original features of the image, inputting the features with anomalies into an image reconstruction decoder and an image restoration decoder respectively, and reconstructing images without anomalies and images with anomalies respectively, the specific steps are as follows: Step 201, feature extraction: The input image x is extracted by a deep feature extractor to obtain features f(x). The deep feature extractor consists of 16 residual blocks. Each residual block contains two 3x3 convolutional layers with a step size of 1 and a 1x1 convolutional layer. Each residual block contains a skip connection, which directly adds the input of the block to the output. Step 202, feature quantization: In order to facilitate the generation of abnormal area data that is closer to the normal distribution, the input image is encoded by vector quantization. Vector quantization is a technology that maps a continuous latent space to discrete features. By introducing a discrete codebook in the latent space, the model can more effectively learn the high-level abstract representation of the data. By introducing a discrete codebook, vector quantization divides the latent space into a finite number of discrete regions, each of which is represented by a specific codebook vector. Specifically, the feature quantization process of the feature f(x) extracted by the deep feature extractor is expressed as: In the above formula (2), f(x) represents the features extracted by the deep feature extractor from the input image x, f(x) ij The feature representing the position of the feature map (i, j), e k represents a vector in the codebook, where k is the index of the codebook vector, and the value range is k∈(1…K), where K is the number of features in the codebook, and argmin represents the e that minimizes the function. k The value of , ||·|| represents the Euclidean distance. Finally, each point in the feature space is replaced by the nearest neighbor vector in the codebook; Step 203, using two-level vector quantization to improve the quality of the reconstruction result: using low-resolution and high-resolution codebooks to perform two-level encoding on the input image, the quantized feature map of the low-resolution codebook is reduced by 4 times compared with the original input image, while the quantized feature map of the high-resolution codebook is reduced by 8 times. Through this two-level encoding method, the multi-level features of the input image can be better captured, and a higher quality reconstruction result can be produced; Step 204, constructing a cloth feature anomaly generation module: in the training stage, the model can only access input images without anomalies, and anomalies need to be generated for the input images. The current noise addition methods are divided into two types: image level and feature level. Among them, the image level noise generation can simulate limited anomalies, while the feature level noise addition is more robust. By introducing noise on specific features of the image, the feature level noise addition can simulate the changes in abnormal features and increase the diversity of abnormal information. While taking into account the randomness of the abnormal area, the model can have better resolution for anomalies that are difficult to distinguish. When performing anomaly generation, two methods of random noise addition and quantized subspace noise addition are combined: random sampling generates a mask Y of the abnormal area, where the area with a value of 1 represents an anomaly, and random noise addition superimposes the generated random noise on the features of the selected area; the quantized subspace noise addition method is based on the quantized features, randomly selecting features from the feature space of the image data to replace the features of the selected area, while limiting the similarity between the replaced features and the original features without anomalies, and excluding the most similar 5% of vectors; Step 205, constructing a cloth image decoder: In the image decoding stage, an image reconstruction decoder and an image restoration decoder are introduced respectively, wherein the image reconstruction decoder aims to restore the abnormal area to the normal appearance of the sample observed during training, while the image restoration decoder directly restores the input features to an image; Step 206, for the image reconstruction decoder: the decoder performs feature extraction through two convolution blocks, wherein the first convolution block contains a 3x3 convolution layer for maintaining the number of input channels, followed by an instance normalization layer and a ReLU activation function, followed by another 3x3 convolution layer to double the number of channels, followed by another normalization layer and a ReLU activation function, the second convolution block operates in a similar manner to the first convolution block, each convolution block is followed by a maximum pooling layer, and both use a 2x2 window and a step size of 2 for downsampling. Next, the feature map passes through a 1x1 convolution layer to reduce the number of channels to 64, and then the feature map passes through two upsampling blocks, each upsampling block uses a 4x4 transposed convolution with a step size of 2 to achieve upsampling operations, and a ReLU activation function is connected after each upsampling block. After that, the feature map passes through a 3x3 convolution layer and then enters two transposed convolution layers to gradually reduce the number of channels for reconstructing the RGB channels of the original image. After each transposed convolution layer, except for the last layer, a ReLU activation function is connected. Finally, the network outputs the reconstructed image; Step 207, for the image restoration decoder: the decoder uses a fixed pre-trained model to achieve direct reconstruction of the input features. The decoder includes a convolution layer for preliminary processing of the potential representation, and extracts deep features through residual stacking. In the residual stacking, each residual block contains two convolution layers with ReLU activation functions, which are upsampled through a deconvolution layer, and then further extracted features through a convolution layer with a ReLU activation function, and then the potential representation is mapped to the space of the original image through another deconvolution layer. The image reconstruction decoder and the image restoration decoder both include operations on input features of two different resolutions. The low-resolution quantized encoded feature map is upsampled and convolved, and then feature extracted through the residual module. The high-resolution feature map is also processed through upsampling and convolution operations. The two feature maps are cascaded together, and the final reconstructed image is generated through transposed convolution.

4. According to the cloth defect detection method of high-frequency feature constraint and edge entropy minimization as claimed in claim 1, the high-frequency feature constraint loss is calculated in step 3 Supervise the quality of the reconstructed de-anomaly image to ensure the reconstruction quality of the detailed texture features. The specific steps are: Step 301, constructing a high-frequency feature constraint module: In an anomaly detection model based on image reconstruction, the higher the quality of the anomaly-free reconstructed image, the stronger the model's understanding and learning ability of normal samples, which helps to improve the robustness and generalization ability of the model, making it more credible in practical applications. When the model is used for anomaly detection tasks, it can better distinguish normal and abnormal samples. The way to enhance the reconstruction quality is to use the Euclidean distance loss to directly constrain the consistency of the anomaly-free reconstructed image with the original image. Since the fitting priority of the neural network for frequency information is different throughout the training process, usually from low to high, this will mean that some high-frequency information is not valued by the network, while the detailed texture information is contained in the high-frequency information; In order to narrow the gap between the high-frequency information of the reconstructed de-anomaly image and the original image, a high-frequency feature constraint module is set up, and the original input image and the reconstructed de-anomaly image are converted to the frequency domain using discrete Fourier transform: In the above formula (3), F(u,v) is the representation of the image in the frequency domain, (u,v) represents the coordinates in the frequency domain, the size of the input image is M×N, x(m,n) represents the pixel value of point (m,n) in the image, e and i are natural constants and imaginary units, respectively, as shown in the following formula (4): In the above formula (4), cos represents cosine, sin represents sine, (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, x(m, n) represents the pixel value of point (m, n) in the image, e and i are natural constants and imaginary units respectively; Step 302, after discrete Fourier transform, the amplitude and phase of the image are two key elements to describe the frequency domain information. The phase describes the spatial position and relative position of the frequency component in the image, determines the starting point of the sine wave in the image, and thus affects the spatial layout of different frequencies in the frequency domain. The amplitude represents the response intensity or energy of the image to a specific frequency. In the centralized amplitude diagram, the low-frequency information is in the middle and the high-frequency information is around. In order to design a loss function that constrains high-frequency information, for the real and imaginary parts of F(u,v) in formula (3), let the real part R(u,v) = a, 6. The imaginary part I(u,v) = b, then F(u,v) is expressed as the following formula (5): F(u,v)=R(u,v)+I(u,v)i=a+bi……(5), In the above formula (5), (u, v) represents the coordinates in the frequency domain, r(u, v) and a represent the real part of the frequency domain information, I(u, v) and b represent the imaginary part of the frequency domain information, and i is the imaginary unit; Step 303: Calculate the amplitude of the image, as shown in equation (6): In the above formula (6), (u, v) represents the coordinates in the frequency domain, R(u, v) and a represent the real part of the frequency domain information, I(u, v) and b represent the imaginary part of the frequency domain information, and |·| represents the modulus of the complex number; Step 304: Calculate the phase of the image according to the following formula (7): In the above formula (7), (u, v) represents the coordinates in the frequency domain, R(u, v) represents the real part of the frequency domain information, I(u, v) represents the imaginary part of the frequency domain information, arctan is the tangent function, and the distance measurement is performed by mapping each frequency value to a Euclidean vector in a two-dimensional space; Step 305: Calculate the spatial frequency value at the input image spectrum coordinate (u, v) according to the following formula (8): F in (u,v)=a in +b in i……(8), In the above formula (8), a in represents the real part of the input image spectrum, b in Represents the imaginary part of the input image spectrum, where i is the imaginary unit; Step 306: Calculate the spatial frequency value at the frequency spectrum coordinate (u, v) of the abnormality-removed reconstructed image according to the following formula (9): F gen (u,v)=a gen +b gen i……(9), In the above formula (9), a gen represents the real part of the spectrum of the image after de-anomalization and reconstruction, b gen represents the imaginary part of the spectrum of the image reconstructed after abnormality removal, i is an imaginary unit; Step 307: Calculate the global distance of the frequency using the square of the Euclidean distance, as shown in the following formula (10): In the above formula (10), and Indicates that from F in (u,v) and F gen The two vectors mapped by (u, v) are based on the definition of amplitude and phase. and Corresponding to the amplitude, the angle θ in and θ gen Corresponding to the phase, ||·|| represents the Euclidean distance, and |·| represents the modulus of the complex number; Step 308: Calculate the frequency distance between the original input image and the de-anomalized reconstructed image, using the average value of the global distance, as shown in the following formula (11): In the above formula (11), F in represents the input image spectrum, F gen represents the spectrum of the image reconstructed by removing abnormalities, (u,v) represents the coordinates in the frequency domain, the size of the input image is M×N, and |·| represents the modulus of the complex number; Step 309: In order to make the model focus more on learning key fine-grained features, filter out low-frequency information, reduce the model's attention to overall structural information, and capture subtle textures and detail features in the image, a mask M is introduced here. r , assuming that the center point of the image is (0, 0), then M r The size of is M×N, the value of the central area rM×rN is 0, and the value of the rest of the area is 1, where r∈(0,1). The mask function is calculated as follows: In the above formula (12), (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, and r is the ratio of the area size of the central region. is an indicator function used to determine whether the frequency domain coordinates (u, v) are outside the specified central area. If they are outside the area, the mask function M r (u,v) takes the value of 1; if it is within the area, it takes the value of 0; Step 310: Calculate the high-frequency feature constraint loss according to the following formula (13): In the above formula (13), (u, v) represents the coordinates in the frequency domain, the size of the input image is M×N, r is the ratio of the area size of the central region, and F in represents the input image spectrum, F gen Represents the spectrum of the image after removing abnormalities and reconstructing it.

5. According to the cloth defect detection method with high-frequency feature constraint and edge entropy minimization of claim 1, the step 4 calculates the difference image between the original input image reconstructed and the reconstructed abnormality-free image to enhance the features of the abnormal area, and the specific steps are: Step 401, constructing a defect feature enhancement module to receive two inputs, namely, a reconstructed original input image and a reconstructed de-anomaly image; Step 402, calculating a difference image between the reconstructed original input image and the reconstructed de-anomaly image, wherein the difference image is obtained by directly subtracting the original input image and the de-anomaly image, so as to highlight the difference area between the images and enhance the features of the abnormal area; Step 403: splice the reconstructed original input image, the reconstructed anomaly-removed image, and the difference image between the two, and input them into a cloth defect anomaly detection module.

6. According to the cloth defect detection method with high-frequency feature constraint and edge entropy minimization of claim 1, in step 5, the reconstructed original input image, the reconstructed de-anomaly image and the difference image between the two are simultaneously input into the anomaly detector, and the anomaly detection loss L is calculated. det , the specific steps are: Step 501, construct a cloth defect anomaly detection module: The function of the anomaly detection module is to generate an anomaly prediction for each pixel point, which is presented in the form of an anomaly score, indicating the degree of difference between the input image and the normal mode. The anomaly detection module accepts three input images, including the reconstructed original input image, the reconstructed anomaly-free image, and the difference image between the above two, and predicts the anomaly score through a deep defect detection network. The deep defect detection network includes an encoder for capturing hierarchical features and a decoder for reconstructing high-resolution anomaly predictions, wherein: The encoder part consists of four convolutional blocks. Each convolutional block includes two 3x3 convolutional layers for feature extraction. The number of output channels of the first convolutional layer is the same as the input, and the number of output channels of the second convolutional layer is twice that of the first one. Each convolutional layer is followed by an instance normalization layer and a ReLU activation function. Each pair of convolutional layers is followed by a 2x2 maximum pooling layer for downsampling, gradually reducing the spatial size of the feature map while increasing the number of feature channels. The decoder part consists of upsampling and convolution blocks, which are used to gradually restore the spatial resolution of the image. Each upsampling step uses bilinear interpolation and is followed by a 3x3 convolution layer to adjust the number of channels, followed by a ReLU activation function. Each upsampling step is followed by a convolution block, which includes two 3x3 convolution layers for further feature refinement. In each upsampling step, the decoder also makes a jump connection with the feature map of the corresponding layer of the encoder. Finally, the output of the decoder is mapped to the required number of output channels through a 1x1 convolution layer. Step 502, calculate the defect detection loss: train by minimizing the anomaly detection loss function between the prediction score P and the anomaly region mask Y. The loss function is expressed as the following formula (14): In the above formula (14), Y i is the true label of the i-th pixel (0 for normal or 1 for abnormal), P i is the predicted value of the i-th pixel, θ is the balance factor used to balance the ratio of positive and negative samples, and γ is the adjustment factor used to adjust the weight of difficult and easy samples.

7. According to the cloth defect detection method with high-frequency feature constraint and edge entropy minimization as claimed in claim 1, the entropy loss is calculated by the edge entropy minimization module in step 6. Optimize the prediction of edge areas with low certainty to improve the model's confidence in the prediction of edge areas. The specific steps are as follows: Step 601, when making pixel-level abnormal region predictions, the model will generate low entropy predictions near the center of the abnormal region and high entropy predictions at the edge of the abnormal region. The abnormal features are usually more significant in the center of the abnormal region, and the model is more likely to learn the common features of these center regions, thereby generating confident predictions for the center region. However, when the model faces the edge of the abnormal region, due to the differences and complexity of the data distribution in this part or insufficient abnormal information, the model cannot accurately capture the detailed information of the edge, resulting in insufficiently confident high entropy predictions in these areas. This shows that only using the supervision mode of the above step 502 still cannot enable the model to generate high-certainty predictions for the abnormal edge region. One way to improve the detection performance of the abnormal detection model in the edge part is to encourage the model to generate high-confidence predictions in the target prediction stage. Here, an edge entropy minimization module is set, and the design uses entropy loss to directly punish low-confidence predictions; Specifically, the edge entropy minimization module does not consider using the generated abnormal area mask Y to improve the edge prediction confidence, but considers a more direct constraint method, that is, to achieve it by minimizing the predicted entropy. According to the Shannon entropy principle, for the input image x, its entropy map is composed of independent pixel-level entropies, and the entropy value of each pixel point is as follows (15): In the above formula (15), Represents the model's abnormal prediction value for the image (m,n) position; Step 602: In order to improve the confidence of the anomaly detection model in the edge uncertainty area, a reweighting mechanism is introduced. Since the predicted values ​​in the edge uncertainty area are all close to 0.5, the distance between the predicted absolute value and 0.5 is calculated, and a greater loss supervision is imposed on the pixels closer to 0.

5. For the (m, n) position of the image, a distance weight is introduced. The following formula (16): In the above formula (16), λ is a hyperparameter that controls the weight decay speed, and its value range is between 0 and 1. The value of λ directly affects the decay speed of the distance weight. The larger the λ is, the faster the decay speed is, and the weight of the prediction value with a longer distance decreases faster; while the smaller the λ is, the slower the decay speed is, and the weight of the prediction value with a longer distance decreases relatively slowly. Step 603: Entropy loss is defined as the sum of all pixel-level entropies, as shown in equation (17): In the above formula (17), Represents the entropy value at the pixel point (m,n), Represents the distance weight at the pixel point (m,n).

8. According to the cloth defect detection method with high-frequency feature constraint and edge entropy minimization as claimed in claim 1, the calculation of the joint loss in step 7 And back propagate to update the parameters. The specific steps are: Step 701: Calculate the joint loss according to the following formula (18): To update the parameters of the image reconstruction decoder and anomaly detector: In the above formula (18), is the anomaly detection loss, is the consistency loss between the abnormal reconstructed image and the original image, is the high-frequency feature constraint loss, is the entropy loss, α and β are and The weight factor of .

9. According to the cloth defect detection method with high-frequency feature constraint and edge entropy minimization of claim 1, in the use stage of the model, the test image is input into the trained cloth defect anomaly detection model, the test image is subjected to defect detection, and the result with a confidence greater than a certain threshold value Φ in the abnormal score result output by the model is selected as the abnormal area. The specific steps are as follows: Step 901: For a defective input image, it first passes through a feature encoder to obtain features with abnormal areas; Step 902: Send the features of the abnormal area to the image reconstruction decoder and the image restoration decoder respectively, and reconstruct the image without the abnormality and the image with the abnormality respectively; Step 903, stitching the generated images with / without anomalies and the difference images of the two together, and inputting them into anomaly detector at the same time; Step 904: predict the abnormal possibility of each pixel point, and the score of the pixel point in the defective area of ​​the final prediction result will be closer to 1, and the score of the pixel point in the non-defective area will be closer to 0; Step 905: For an input image without defects, after the same reasoning process, the defect anomaly score of the entire image area will be close to 0.

Images