A system and method for detecting appearance defects of ready-made clothing based on image semantics

Abstract

The present invention discloses a system and method for detecting appearance defects of garments based on image semantics, comprising a semantic extraction module, a semantic deep compression sampling module, an image reconstruction and denoising module and an image classification module. The semantic extraction module uses image semantics to capture semantic features directly related to the appearance defects of garments, thereby extracting a set of M important semantic blocks M<16 that contribute most to the accuracy of the classification model. A reinforcement learning feedback mechanism is introduced to optimize the model strategy. The semantic block deep compression sampling module uses a learnable convolution kernel to compress and sample the image. The image restoration and denoising module uses a restoration matrix to restore and denoise the compressed image through a learnable transposed convolution kernel. The classifier module is the output module of the system, and the processed image is sent to a CNN classifier. The present invention reduces the sensitivity to noise and interference factors in the image by extracting semantic features from the garment appearance defect image, thereby improving the anti-interference ability of the system.

Description

Technical Field

[0001] The present invention relates to a system and method for detecting appearance defects of ready-made clothes based on image semantics, and belongs to the cross-technical field of communication and artificial intelligence technology. Background Art

[0002] With the development of industrial automation production technology, various industries have begun to put advanced industrial inspection equipment into use to improve production efficiency and ensure the quality of automated production. In the actual production process of garments, due to the influence of various factors, some appearance defects often occur, such as thread ends, spots, collar size, button position and wrinkles. These defects lead to imperfect overall appearance of garments, affecting the texture and grade of the product and consumer experience. In order to increase sales and provide a good consumer experience, the textile industry must strictly control the appearance defects of garments and adopt advanced quality inspection technology and equipment to ensure that the appearance quality of textile garments meets the expectations and requirements of consumers. However, traditional manual quality inspection is time-consuming and costly, and manual visual inspection is affected by objective factors such as light, the surface texture of the garment itself or some subjective emotions. Sometimes it is difficult to accurately identify defective products, which greatly reduces the efficiency of quality inspection. Therefore, the automatic detection technology of garment appearance defects has attracted widespread attention from academia and industry.

[0003] In recent years, the methods of image defect detection and classification mainly include the following: machine learning-based methods, feature extraction-based methods and deep learning-based methods. Commonly used image classification techniques in traditional machine learning methods include support vector machine (SVM), decision tree and random forest. Taking the binary classification model support vector machine as an example, its basic idea is to maximize the distance from the nearest sample point to the hyperplane by finding a hyperplane that can best separate two different categories. In the image classification task, SVM can map the image to a high-dimensional feature space and find an optimal separation hyperplane in the feature space to achieve the classification task. The paper "An image classification method by combining with improved edit distance and SVM" by Y. Wei, J. Tian, ​​R. Guo et al. (Application of Electronic Technique, 2017, 43 (8): 127-131) proposes an image classification method that combines multi-scale string representation of images with improved edit distance and SVM classifier. It uses string combination to describe multi-scale images, and measures the similarity of two images by the edit distance between strings as the basis for image classification. The above methods have certain application value in some small-scale image classification tasks. However, with the development of deep learning and the advantages of deep neural networks, deep learning, especially convolutional neural networks (CNN), has achieved better performance in image classification tasks. The paper "Deep residual learning for image recognition" (IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 770-778) by K. He, X. Zhang, S. Ren et al. proposed the ResNet residual network, which introduces jump connections and residual modules to train deeper networks. Among them, the jump connection allows information to propagate directly across layers in the network, thereby alleviating the gradient disappearance problem, and the residual module consists of two convolutional layers and jump connections, and learns the increment of features by performing residual mapping on the input. ResNet has higher accuracy than traditional deep networks, and even when the number of network layers reaches 152, ResNet can still achieve better performance.To further enhance the accuracy and robustness of image classification, the paper "Improved Leaf Image Classification Using Topological Features and CNN With Attention Module" (IEEE 6th International Conference on Signal and Image Processing (ICSIP), pp. 311-315, 2021) by Y. Zhang et al. combines topological features and the attention module of convolutional neural networks to propose an improved method to enhance the performance of leaf image classification. Specifically, this method first preprocesses the leaf image through computer vision technology to extract topological features to describe the leaf morphology and texture information. Then, a deep convolutional neural network model is constructed, which consists of multiple convolutional layers and pooling layers to learn high-level feature representations of leaf images. Finally, the attention mechanism is introduced. By using the attention module, the network can automatically learn and emphasize important leaf areas, thereby improving the classification performance.

[0004] In general, image classification methods based on deep learning have shown obvious superiority, but none of the above image classification methods use semantic block extraction and reinforcement learning feedback mechanism. The image semantics-based garment appearance defect binary classification method proposed in the present invention belongs to the category of deep learning-based methods. It uses a deep learning model to learn the semantic information in the image and extract the most representative semantic feature blocks in the image, such as the collar, buttons, texture and other parts of the garment, so as to realize image defect detection and classification. Summary of the invention

[0005] The purpose of the present invention is to propose a method for detecting appearance defects of ready-made clothing based on image semantics in response to the defects or shortcomings of the above-mentioned prior art. The method combines the advantages of technologies such as image semantic extraction, compressed sampling, and reinforcement learning feedback to optimize the generalization ability of the model and improve the accuracy of image classification to a certain extent, thereby effectively solving the problem of low efficiency of manual quality inspection.

[0006] The technical solution adopted by the present invention to solve its technical problem is: a garment appearance defect detection system based on image semantics, the system includes a semantic extraction module, a semantic block deep compression sampling module, an image reconstruction and denoising module and an image classification module.

[0007] The semantic extraction module is used to extract the M image block sets (M<16) in the textile garment images that contribute most to the accuracy of the back-end binary classification model, and introduces a reinforcement learning feedback mechanism to optimize the policy network. The output of the reinforcement learning policy network is a 16-dimensional vector. The probability of each semantic block being selected is obtained through Bernoulli mapping, and then the important semantic blocks are selected through Bernoulli sampling, which is the basis for subsequent modules.

[0008] The semantic block deep compression sampling module takes the 0-1 matrix obtained by the semantic extraction module as input, which can be regarded as a linear signal. The module uses a learnable convolution kernel to map the linear signal to the compression domain and uses a sampling matrix to perform block-by-block sampling. The sampling rate is 0.3, and after sampling, it is quantized with 8 bits and input into the channel.

[0009] The image restoration and denoising module uses a learnable transposed convolution kernel to restore the above compressed and quantized vectors into original semantic blocks, and inputs them into the denoising network for denoising.

[0010] The image classification module is the output module of the system, which takes the restored and denoised textile garment images as input. This module uses the classification network, which is CNN, including components such as convolutional layers and fully connected layers, and adopts softmax multi-class classifier and multi-class cross entropy loss function to detect garment appearance defect images.

[0011] The present invention also provides a method for detecting appearance defects of ready-made clothing based on image semantics, the method comprising the following steps:

[0012] Step 1: Dataset acquisition and preprocessing: 680 images of defective garments and 680 images of normal garments were selected and preprocessed.

[0013] Step 2: Construct a reinforcement learning policy network. The policy network takes the original image as input, outputs a 16-dimensional vector, uses the sigmoid function to activate the output, and converts the output into a probability distribution between 0 and 1, during which reinforcement learning feedback is introduced.

[0014] Step 3: Use Bernoulli mapping and sampling. According to the probability output in the previous step, use Bernoulli mapping to convert it into a binary random number between 0 and 1, and then sample the Bernoulli distribution to obtain a 0-1 matrix, which represents the selected semantic block.

[0015] Step 4: Semantic block compression sampling: Sample the above semantic block to obtain a vector, which is quantized and input into the channel.

[0016] Step 5: Receiver recovery and denoising: The received vector is restored into semantic blocks and denoised by a denoising network.

[0017] Step 6: The classifier performs binary classification and divides the garment appearance images into two categories: defective and non-defective.

[0018] Step 7: Test the model effect.

[0019] Furthermore, the above step 1 of the present invention includes: selecting a data set of 680 pictures of the appearance of defective and flawless ready-made clothes for an image classification task, wherein the ready-made clothes include different patterns and different types, such as suits, shirts, jackets, etc., and then preprocessing the images, fixing the pixel size of the images to 256*256, converting them into the Tensor format in PyTorch, and performing a normalization operation on them to facilitate the subsequent training of the deep learning model.

[0020] Furthermore, after the data set for training the model is obtained in the above step 1 of the present invention, the present invention reads the textile garment appearance image data set as the input of the semantic extraction strategy network. First, a strategy network is established, which is a three-layer neural network composed of a feature extraction module, a gain module and a decision module. Specifically, the feature extraction module uses a pre-trained ResNet18 model except the last layer to extract the features of the input image. The gain module and the decision module are both self-designed two-layer fully connected neural networks, in which the gain module is used to convert a single gain value into a 512-dimensional vector, and the decision module is used to splice the feature vector and the gain vector together to generate an N-dimensional vector as the output of the strategy network. Then, the sigmoid function is used to activate the output, and the output is converted into a probability distribution between 0-1, indicating the probability of each semantic block being selected in the N semantic blocks. The process can be expressed as:

[0021] p = sigmoid(f ψ (f θ (X lr )+fφ(g)))

[0022] Among them, X lr , g are the input image and channel gain respectively. θ is the ResNet18 model for image feature extraction, and f φ and f ψ is a multilayer perceptron (MLP), θ, ψ represents the corresponding parameters of these components, respectively, and the sigmoid nonlinear activation function.

[0023] The present invention also specially constructs a reward function, which is called in the policy network training. The function calculates the reward value according to some actual parameters (such as denoised images, classification accuracy, etc.), and uses the reward value as a feedback signal to update the parameters of the policy network, thereby optimizing the training strategy to adapt to the appearance images of ready-made clothes with different defect types, sizes and positions.

[0024] Furthermore, the above step 3 of the present invention includes: taking the 0-1 matrix obtained in the above step 2 as input, defining 16 Bernoulli distributions for the formula p in step 2, and assuming that p[i] represents the probability of being extracted as a semantic block in each dimension i, and the action performed by the policy network can be in each dimension a[i]~{0,1}p [i] Upsampling, where a[i] = 1 means extracting the i-th semantic block for compression, otherwise a[i] = 0, is sampling the Bernoulli distribution to obtain a 0-1 matrix. Among them, the joint probability density function of the policy network output can be expressed as:

[0025]

[0026] Furthermore, the above step 4 of the present invention includes: mapping the 0-1 matrix obtained in the above step 3 to the compression domain for sampling. For high-dimensional image signals, the sampling process is performed block by block, so the sampling process can be expressed as:

[0027]

[0028] in is the vector form of the ith sub-block in the pixel domain, is x i The compressed signal, Represents the sampling matrix, and a learnable convolution kernel is used in the present invention. Specifically, Is an image block with channels as RGB The reconstructed vector, where k represents the height and width of the sampling sub-block, and 3 represents the RGB channel. In the present invention, the convolution layer converts the input image X=stitch{X 1 , X 2 , …, X P} and the convolution kernel κ to obtain the output image Y = stitch {Y 1 , Y 2 , …, Y P}, the process can be expressed as:

[0029] Y=κ*X

[0030] in, represents the unbiased convolution kernel, sr represents the sampling rate, * represents the convolution operation with a step size of k, stitch represents stitching all image blocks into a complete image, and P represents the total number of image blocks.

[0031] Furthermore, the above step 5 of the present invention includes: restoring the compressed image vector obtained in the above step 4 into a semantic block. Traditional algorithms such as the denoising-based approximate message passing algorithm (D-AMP) restore compressed images by alternating iterative residual variables and reconstructed signals. The residual correction term of the algorithm requires the use of complex divergence calculations. In order to reduce the computational complexity, the present invention uses a convolutional neural network with a residual structure to replace the residual correction term. Specifically, assuming that the residual consists of L iterations, Represents the reconstructed image output of the previous stage, compressed domain Each sub-block in can be calculated by the sampling matrix. Then, the residual is mapped back to the remaining pixel domain Correction is performed and the residual is calculated using the following formula:

[0032]

[0033] The residual is further processed using a convolutional neural network to obtain the updated residual. The update process can be expressed as:

[0034]

[0035] Among them, ω l is the parameter of the two-layer convolutional residual network, Z l represents the residual after the update in the lth stage. Finally, a three-layer convolutional denoising network is used to transform the residual Z in the pixel domain l Restore to the original image size and dimensions, that is, use the following formula to restore:

[0036]

[0037] It should be noted that since only the semantic part of the image is transmitted and the reconstruction process is performed on the entire image, the present invention sets the compression value of the non-semantic part to 0 in step 3 to reduce the computational complexity of the reconstruction network.

[0038] Furthermore, the above step 6 of the present invention includes: inputting the restored image after denoising in the above step 5 into the classifier for classification. The classifier uses a pre-trained ResNet152 neural network model, which acts as a feature extractor, and the number of output features is set to the number of n_classes categories. The present invention sets n_classes to 2 to complete the task of classifying the image.

[0039] Furthermore, the above step 7 of the present invention includes: randomly selecting 90% of the data as a training set, and the remaining 10% of the data as a training set and a test set to adjust the model hyperparameters and test the performance of the model in terms of classification accuracy (Accuracy), reward value (Reward) and other indicators.

[0040] Beneficial effects:

[0041] 1. In the policy-based reinforcement learning method, the present invention uses a convolutional neural network-based model to extract image semantic feature blocks. The semantic feature blocks extracted by the model can better represent the structure and content of the garment appearance image, and can capture the local details, texture, shape and other information of the defects in the defect detection and classification tasks, thereby improving the representation ability of image features. Compared with the traditional image classification model, it has achieved a significant improvement in the accuracy evaluation index.

[0042] 2. The present invention also introduces a reinforcement learning feedback mechanism and constructs a reward function get_reward(). The reward function calculates the reward value based on some actual parameters such as the denoised image and classification accuracy, and uses the reward value as a feedback signal to update the parameters of the policy network, thereby optimizing the training strategy. This method can adaptively perform defect detection and classification on garment data sets of different colors and types, improving the generalization ability and adaptability of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The figure is a flow chart of the method of the present invention.

[0044] Figure 2 It is a schematic diagram of the simulation results of the present invention based on the clothing appearance defect detection data set.

[0045] Among them, (2a) and (2b) are schematic diagrams of the accuracy curve and reward curve of the present invention on the training set and the verification set. DETAILED DESCRIPTION

[0046] The invention will be described in further detail below in conjunction with the accompanying drawings.

[0047] It should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0048] like Figure 1 As shown, the present invention provides a method for detecting appearance defects of ready-made clothing based on image semantics, the method comprising the following steps:

[0049] Step 1: Dataset acquisition and preprocessing.

[0050] In the present invention, firstly, a data set of defective and flawless images is selected for image classification tasks. Taking binary classification as an example, 680 images of defective and flawless garment appearance are selected and stored in defected and normal folders respectively. The image data of the data set covers various important defects of garments in the textile industry, including garment appearance images of different defect types, sizes and positions, as well as different patterns (such as plain cloth and patterned cloth) and different types (such as suits, shirts, jackets, etc.). Secondly, the present invention preprocesses the image, fixes the pixel size of the image to 256*256, converts it into the Tensor format in PyTorch, and performs normalization operations on it, so as to facilitate the subsequent deep learning model training.

[0051] Step 2: Build a reinforcement learning policy network.

[0052] After obtaining the data set for training the model according to the above step 1, the present invention reads the data set as the input of the policy network. First, a policy network is established, which is a three-layer neural network composed of a feature extraction module, a gain module and a decision module. Specifically, the feature extraction module uses a pre-trained ResNet18 model except the last layer to extract the key features of the input garment appearance image. The gain module and the decision module are both self-designed two-layer fully connected neural networks, in which the gain module is used to convert a single gain value into a 512-dimensional vector, and the decision module is used to splice the feature vector and the gain vector together to generate an N-dimensional vector as the output of the policy network (N is 16 in the present invention). Then, the sigmoid function is used to activate the output, and the output is converted into a probability distribution between 0-1, indicating the probability of each semantic block being selected in the N semantic blocks. The process can be expressed as:

[0053] p = sigmoid(f ψ (f θ (X lr )+f φ (g)))

[0054] Among them, X lr , g are the input clothing appearance image and channel gain respectively. θ is the ResNet18 model for image feature extraction, and f φ and f ψ is a multilayer perceptron (MLP), θ, ψ represents the corresponding parameters of these components, respectively, and the sigmoid nonlinear activation function.

[0055] The present invention also specially constructs a reward function, which is called in the policy network training. The function calculates the reward value according to some actual parameters (such as denoised images, classification accuracy, etc.), and uses the reward value as a feedback signal to update the parameters of the policy network, thereby optimizing the training strategy.

[0056] Step 3: Use Bernoulli mapping and sampling.

[0057] In order to allow the policy network to flexibly select image blocks in any image area, the present invention divides the image into a grid shape, and each block is determined by a separate layer to decide whether it needs to be transmitted. Figure 1 As shown in Figure 2, assume that the original image is divided into 4×4 blocks, that is, N in step 2 is 16, where each block provides different semantic information for the backend model, and the semantic extraction module can select any combination of them. In the formula in step 2, p defines 16 Bernoulli distributions, and p[i] represents the probability of being extracted as a semantic block in each dimension i. The actions performed by the policy network can be in each dimension a[i]~{0,1}p [i] Upsampling, where a[i] = 1 means extracting the i-th semantic block for compression, otherwise a[i] = 0, is sampling the Bernoulli distribution to obtain a 0-1 matrix. Therefore, the joint probability density function of the policy network output can be expressed as:

[0058]

[0059] Step 4: Semantic block compression sampling.

[0060] The present invention maps the 0-1 matrix obtained in step 3 to the compression domain for sampling through a sampling matrix. For high-dimensional image signals, the sampling process is performed block by block, so the sampling process can be expressed as:

[0061]

[0062] in is the vector form of the ith sub-block in the pixel domain, is x i The compressed signal, Represents the sampling matrix, and a learnable convolution kernel is used in the present invention. Specifically, It is an image block with channels RGBB The reconstructed vector, where k represents the height and width of the sampling sub-block, and 3 represents the RGB channel. In the present invention, the convolution layer converts the input image X=stitch{X 1 , X 2 , …, X P} and the convolution kernel κ to obtain the output image Y = stitch {Y1 , Y 2 , …, Y P}, the process can be expressed as:

[0063] Y=κ*X

[0064] in, represents the unbiased convolution kernel, sr represents the sampling rate, * represents the convolution operation with a step size of k, stitch represents stitching all image blocks into a complete image, and P represents the total number of image blocks.

[0065] Step 5: Receiver recovery and denoising.

[0066] The present invention will restore the image vector compressed in step 4 into a semantic block. Traditional algorithms such as the denoising-based approximate message passing algorithm (D-AMP) restore compressed images by alternating the residual variables and reconstructing the signal. The residual correction term of the algorithm requires complex divergence calculation. In order to reduce the computational complexity, the present invention uses a convolutional neural network with a residual structure to replace the residual correction term. Specifically, assuming that the residual consists of L iterations, Represents the reconstructed image output of the previous stage, compressed domain Each sub-block in can be calculated by the sampling matrix. Then, the residual is mapped back to the remaining pixel domain Correction is performed and the residual is calculated using the following formula:

[0067]

[0068] The residual is further processed using a convolutional neural network to obtain the updated residual. The update process can be expressed as:

[0069]

[0070] Among them, ω l is the parameter of the two-layer convolutional residual network, Z l represents the residual after the update in the lth stage. Finally, a three-layer convolutional denoising network is used to transform the residual Z in the pixel domain l Restore to the original image size and dimensions, that is, use the following formula to restore:

[0071]

[0072] It should be noted that since only the semantic part of the image is transmitted and the reconstruction process is performed on the entire image, the present invention sets the compression value of the non-semantic part to 0 in step 3 to reduce the computational complexity of the reconstruction network.

[0073] Step 6: The classifier performs binary classification. The present invention inputs the restored image after denoising in step 5 into the classifier for classification. The classifier uses a pre-trained ResNet152 neural network model, which acts as a feature extractor, and the number of output features is set to the number of n_classes categories. The present invention sets n_classes to 2 to complete the image classification task.

[0074] Step 7: Test the model effect.

[0075] To test the model effect, the present invention randomly selects 90% of the garment appearance defect detection data set as the training set, and the remaining 10% of the data as the training set and the test set to adjust the model hyperparameters. The present invention tests the performance of the model in terms of classification accuracy (Accuracy), reward value (Reward) and other indicators.

[0076] The effects of the present invention are further described in detail below in conjunction with simulation experiments.

[0077] 1. Simulation conditions and parameter settings:

[0078] The simulation experiment of the present invention is carried out on the simulation platform of Python3.8 and pytorch1.15. The computer CPU model is AMD EPYC7302, and the GPU model is NVIDIA Geforce RTX 3090. The clothing appearance defect image dataset used in the present invention is derived from a web crawler, the learning rate is set to 0.00001, the weight decay parameter is set to 0.0001, and the batch size is set to 16.

[0079] 2. Simulation content:

[0080] Figure 2 The figure shows the performance of the model used in the technical solution of the present invention in terms of classification accuracy and reward value based on the clothing appearance defect detection dataset. The dataset used in the simulation contains clothing appearance defect images of different defect types, sizes and locations, and there is also noise interference. Figure 2 The horizontal axis of the simulation results shown is epoch, which represents the number of iterations of the training model. Figure 2 (a), the vertical axis is the accuracy of the two-class classification Acc, Figure 2 (b) The vertical axis is the reward value.

[0081] Figure 2(a) shows the effect of the present invention on the binary classification of the garment appearance defect detection data set. The training process was carried out for 100 rounds, and the figure shows the result curve of the accuracy on the training set and the validation set. It can be observed from the figure that as the number of training rounds increases, the accuracy of the model in the training set gradually increases, and the accuracy on the validation set averages about 95%, which shows that the present invention has good learning ability and generalization ability for the garment appearance defect classification task. Figure 2 (b) shows the reward curve obtained by the reinforcement learning model developed by the present invention based on the garment appearance defect detection dataset. It can be observed from the figure that the reward shows a convergence trend after 80 rounds of training on the training set, indicating that the model is gradually optimized during training and learns the best strategy. The reward on the validation set also shows a trend of slowly rising to a stable state, indicating that the model is gradually optimized on the validation set and can be generalized to a certain extent.

[0082] Table 1

[0083]

[0084]

[0085] Table 1 shows a comparison table of classification accuracy before and after training based on the garment appearance defect detection dataset using the method of the present invention. The table shows that the classification accuracy of the validation set using only the ResNet152 neural network classification model is about 85%, however, the classification accuracy after training by applying the method of the present invention is significantly improved to 95%, an increase of 10% over the previous one.

[0086] Based on the above simulation results and analysis, the feasibility and effectiveness of the garment appearance defect detection method based on image semantics proposed in the present invention enable the present invention to be better applied in actual recommendation scenarios.

[0087] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the claims attached to the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the attached claims, or the equivalent forms of such scope and boundaries.

Claims

1. A garment appearance defect detection system based on image semantics, characterized in that: The system includes a semantic extraction module, a semantic block deep compression sampling module, an image reconstruction and denoising module, and an image classification module; The semantic extraction module is used to extract the M image blocks (M<16) that contribute most to the accuracy of the back-end binary classification model in the textile garment image, and introduces a reinforcement learning feedback mechanism to optimize the strategy network, where the output of the reinforcement learning strategy network is a 16-dimensional vector. The probability of each semantic block being selected is obtained through Bernoulli mapping, and then the important semantic blocks are selected through Bernoulli sampling. The semantic block deep compression sampling module, image restoration and denoising module and image classification module are the basis of the system's output module; The semantic block deep compression sampling module takes the 0-1 matrix obtained by the semantic extraction module as input, which can be regarded as a linear signal. The module uses a learnable convolution kernel to map the linear signal to the compression domain and uses a sampling matrix to perform block-by-block sampling. The sampling rate is 0.3, and after sampling, it is quantized with 8 bits and input into the channel. The image restoration and denoising module uses a learnable transposed convolution kernel to restore the above compressed and quantized vectors into the original semantic blocks, and inputs them into the denoising network for denoising; The image classification module is the output module of the system. It takes the restored and denoised textile garment images as input. This module uses the classification network, which is CNN, including convolutional layer and fully connected layer components, and adopts softmax multi-category classifier and multi-class cross entropy loss function to realize the detection of garment appearance defect images.

2. A method for detecting appearance defects of ready-made clothing based on image semantics, characterized in that: The method comprises the following steps: Step 1: Dataset acquisition and preprocessing: 680 images of defective garments and 680 images of normal garments were selected and preprocessed; Step 2: Construct a reinforcement learning policy network, which takes the original image as input, outputs a 16-dimensional vector, uses the sigmoid function to activate the output, converts the output into a probability distribution between 0 and 1, constructs a reward function, and calls the reward function in the policy network training. The function calculates the reward value based on some actual parameters and uses the reward value as a feedback signal to update the parameters of the policy network, thereby optimizing the training strategy to adapt to garment appearance images with different defect types, sizes, and locations; Step 3: Use Bernoulli mapping and sampling. According to the probability output in the previous step, use Bernoulli mapping to convert it into a binary random number of 0-1. Then sample the Bernoulli distribution to obtain a 0-1 matrix to represent the selected semantic block. Step 4: semantic block compression sampling, sampling the above semantic block to obtain a vector, which is quantized and input into the channel; Step 5: Recovery and denoising at the receiving end: restore the received vector into semantic blocks and denoise it through a denoising network; Step 6: The classifier performs binary classification and divides the garment appearance image into two categories: defective and non-defective; Step 7: Test the model effect.

3. The method for detecting garment appearance defects based on image semantics according to claim 2, characterized in that: Step 1 of the method includes: selecting a data set of 680 images of the appearance of defective and flawless ready-made clothes for an image classification task, wherein the ready-made clothes include different patterns and different types, and then preprocessing the images, fixing the pixel size of the images to 256*256, converting them into a Tensor format in PyTorch, and performing a normalization operation on them to facilitate subsequent deep learning model training.

4. The method for detecting garment appearance defects based on image semantics according to claim 3, characterized in that: Step 2 of the method comprises: after obtaining the data set for training the model according to the above step 1, reading the textile garment appearance image data set as the input of the semantic extraction strategy network, firstly establishing the strategy network, the strategy network is a three-layer neural network composed of a feature extraction module, a gain module and a decision module, the feature extraction module uses a pre-trained ResNet18 model except the last layer to extract the features of the input image, the gain module and the decision module are both self-designed two-layer fully connected neural networks, wherein the gain module is used to convert a single gain value into a 512-dimensional vector, and the decision module is used to splice the feature vector and the gain vector together to generate an N-dimensional vector as the output of the strategy network, and then, using the sigmoid function to activate the output, converting the output into a probability distribution between 0 and 1, indicating the probability of each semantic block in the N semantic blocks being selected, specifically expressed as: p=sigmoid(f ψ (f θ (X lr )+f φ (g))) Among them, X lr , g are the input image and channel gain respectively, f θ is the ResNet18 model for image feature extraction, and f φ and f ψ is a multilayer perceptron (MLP), θ, ψ represents the corresponding parameters of these components, respectively, and the sigmoid nonlinear activation function.

5. The method for detecting garment appearance defects based on image semantics according to claim 4, characterized in that: Step 3 of the method comprises: taking the 0-1 matrix obtained in step 3 as input, formula p defines 16 Bernoulli distributions, and assuming that p[i] represents the probability of being extracted as a semantic block in each dimension i, and the action performed by the policy network is in each dimension a[i]~{0,1} p[i] Upsampling, where a[i]=1 means extracting the i-th semantic block for compression, otherwise a[i]=0, is sampling the Bernoulli distribution to obtain a 0-1 matrix, where the joint probability density function of the policy network output is expressed as:

6. The method for detecting garment appearance defects based on image semantics according to claim 2, characterized in that: Step 4 of the method includes: mapping the 0-1 matrix obtained in step 3 to the compression domain for sampling. For high-dimensional image signals, the sampling process is performed block by block, so the sampling process is expressed as: in is the vector form of the ith sub-block in the pixel domain, is x i The compressed signal, Represents the sampling matrix, using a learnable convolution kernel, Is an image block with channels as RGB The reconstructed vector, where k represents the height and width of the sampling sub-block, 3 represents the RGB channel, and the convolution layer converts the input image X = stitch {X1, X2, ···, X P } and the convolution kernel κ to obtain the output image Y = stitch{Y1,Y2,…,Y P }, the process is expressed as: Y=κ★X in, represents the unbiased convolution kernel, sr represents the sampling rate, ★ represents the convolution operation with a step size of k, stitch represents stitching all image blocks into a complete image, and P represents the total number of image blocks.

7. The method for detecting garment appearance defects based on image semantics according to claim 2, characterized in that: Step 5 of the method comprises: restoring the compressed image vector obtained in step 4 into a semantic block, using a convolutional neural network with a residual structure to replace the residual correction term, assuming that the residual consists of L iterations, and in each iteration Represents the reconstructed image output of the previous stage, compressed domain Each sub-block in is calculated by the sampling matrix, and the residual is mapped back to the remaining pixel domain Correction is performed and the residual is calculated using the following formula: The residual is further processed using a convolutional neural network to obtain the updated residual. The update process is expressed as: Among them, ω l is the parameter of the two-layer convolutional residual network, Z l Represents the residual after the update in the first stage. Finally, a three-layer convolutional denoising network is used to transform the residual Z in the pixel domain l Restore to the original image size and dimensions, that is, use the following formula to restore: In the above step 3, the compression value of the non-semantic part is set to 0 to reduce the computational complexity of the reconstruction network.

8. The method for detecting garment appearance defects based on image semantics according to claim 2, characterized in that: Step 6 of the method includes: inputting the restored image after denoising in the above step 5 into the classifier for classification, the classifier uses a pre-trained ResNet152 neural network model, which serves as a feature extractor, and the number of output features is set to the number of n_classes categories, and n_classes is set to 2 to complete the image classification task.

9. The method for detecting garment appearance defects based on image semantics according to claim 2, characterized in that: Step 7 of the method includes: randomly selecting 90% of the data as a training set and the remaining 10% of the data as a training set and a test set to adjust the model hyperparameters and test the performance of the model in terms of classification accuracy and reward value.

Images