Intelligent detection and classification system for knitting fabric defects based on deep learning

Abstract

The application relates to the technical field of textile detection, and discloses a knitted fabric defect intelligent detection and classification system based on deep learning, which aims to solve the problems of low adaptability and low detection precision of the existing detection method for diversified defects. The system first decomposes a fabric image into a texture primitive matrix, and innovatively adopts a local and global dual-channel parallel verification strategy: the local verification calculates a structural residual error through a context generation model to identify microscopic structural abnormalities; the global verification calculates a syntax abnormality score through a texture grammar model to identify macroscopic organization errors. Then, an information fusion module combines the two verification results to generate a comprehensive abnormality response map, and finally a defect decision module realizes accurate positioning and classification of the defects. Through the collaborative analysis of local and global information and the use of the unsupervised learning paradigm, the core model training can be completed without defect samples, and the detection precision and robustness for various fabric defects are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of textile detection, in particular to a knitted fabric defect intelligent detection and classification system based on deep learning. BACKGROUND

[0002] In the production process of knitted fabric, due to the influence of factors such as equipment state, raw material quality and process parameter fluctuation, various types of surface defects such as broken warp, broken weft, oil stain and hole will inevitably occur. These defects directly affect the final quality and grade of the fabric, so comprehensive quality inspection of the fabric before leaving the factory is an indispensable link in the production process.

[0003] Traditional defect detection mainly relies on manual visual inspection, which not only has high labor intensity and slow detection speed, but also the detection result is easily affected by subjective factors (such as the experience and fatigue degree of the inspector), resulting in non-uniform detection standards and frequent missed detection and false detection. In order to overcome the above problems, automatic detection technology based on machine vision is introduced into this field.

[0004] Early automatic detection methods are mostly based on traditional image processing and statistical models. However, these methods have limited feature extraction capability when facing knitted fabrics with complex texture and various defect shapes, and it is difficult to establish a detection model with sufficient generalization ability.

[0005] With the development of deep learning technology, defect detection methods based on supervised learning, such as using convolutional neural networks for classification or segmentation, have shown high detection accuracy. However, the performance of such methods is highly dependent on large-scale, diverse and accurately labeled defect sample datasets. In actual industrial production, obtaining and labeling samples covering all defect types not only consumes huge human and time costs, but also for occasional and unknown new defects, such models cannot effectively identify them, limiting their applicability on rapidly changing production lines.

[0006] To reduce the dependence on defective samples, some studies have turned to the paradigm of unsupervised learning-based anomaly detection. Such methods construct a normal model by learning the feature distribution of a large number of normal samples, and identify the areas deviating from the norm as defects. However, existing technologies still have deficiencies in implementation. Some methods only focus on the local area texture consistency of fabric images, and judge the anomaly by comparing the reconstruction or prediction residual of image blocks. Although this strategy is sensitive to small structural defects, it lacks global constraints and misses some slowly changing defects that involve the overall organization of the fabric (such as sparse weft and dense path). Other methods focus on global statistical features of images, but ignore local subtle texture damage. Existing technologies lack effective solutions to combine local fine structure analysis with global organization rule checking. When faced with complex defects with both local and global characteristics, the detection performance is limited. In addition, existing methods usually rely only on a single dimension of difference measure (such as L2 norm in pixel space) when quantifying local anomalies. This approach is prone to misjudging normal and harmless texture fluctuations (such as uneven yarn thickness) in the fabric as defects, resulting in a high false positive rate. SUMMARY

[0007] The technical problem to be solved by the present application is that existing fabric defect detection technology has low detection accuracy, high miss rate and high false positive rate when facing weak, complex and both local texture anomaly and global structure error defects.

[0008] To solve the above technical problems, the present application provides a kind of based on deep learning's fabric defect intelligent detection and classification system, including:

[0009] Texture primitive extraction module, for the input fabric image is decomposed into two-dimensional primitive matrix composed of multiple texture primitives;

[0010] Local checking module, which is connected with the texture primitive extraction module, the local checking module is configured with a local generative model, the local generative model is a deep learning model trained from normal fabric samples to learn local weaving rules, the local checking module uses the local generative model to generate a predicted texture primitive according to the context of the target texture primitive, and calculates the structural residual value between the predicted texture primitive and the target texture primitive, and then generates a structural residual map;

[0011] a global checking module connected with the texture primitive extraction module, the global checking module being configured with a global texture syntax model, the global texture syntax model being a deep learning model trained via normal fabric samples to learn fabric global organization syntax, the global checking module using the global texture syntax model to analyze the texture primitive sequence converted from the two-dimensional primitive matrix to calculate a syntax anomaly score and further generate a syntax anomaly map;

[0012] an information fusion module connected with the local checking module and the global checking module, the information fusion module fusing the structural residual map and the syntax anomaly map to generate a comprehensive anomaly response map;

[0013] a defect decision module connected with the information fusion module, the defect decision module locating and classifying defect regions in the fabric image according to the comprehensive anomaly response map.

[0014] In a preferred embodiment, the local generation model is a conditional generative adversarial network. The conditional generative adversarial network comprises a generator configured to receive a context of the target texture primitive as a conditional input to generate the predicted texture primitive.

[0015] In a specific embodiment, the structural residual value is calculated based on differences between the predicted texture primitive and the target texture primitive in at least two or more feature spaces: pixel space, frequency space, and gradient space. Further, the structural residual value is calculated by a multi-modal residual fusion formula configured to quantify and weighted-sum the differences in the pixel space, the frequency space, and the gradient space. The formula is specifically expressed as:

[0016] ;

[0017] wherein:

[0018] represents the final calculated structural residual value;

[0019] represents a representation of the target texture primitive in the pixel space, represents a representation of the predicted texture primitive in the pixel space;

[0020] represents a representation of the target texture primitive in the frequency space, represents a representation of the predicted texture primitive in the frequency space;

[0021] represents a representation of the target texture primitive in the gradient space, represents a representation of a predicted texture primitive in gradient space;

[0022] respectively represent distance metric functions for computing the difference between two inputs in pixel space, frequency space, and gradient space, respectively;

[0023] respectively represent preset fusion weights for corresponding pixel space, frequency space, and gradient space residuals;

[0024] represents a residual value in pixel space;

[0025] represents a weighted pixel space residual;

[0026] represents a residual value in frequency space;

[0027] represents a weighted frequency space residual;

[0028] represents a residual value in gradient space;

[0029] represents a weighted gradient space residual.

[0030] In one preferred embodiment, the global texture grammar model is a variational autoencoder-based context-free grammar model. The model includes an encoder and a decoder, which are used to compute the grammar anomaly score by probabilistically modeling the sequence of texture primitives. Further, the probabilistic modeling is used to determine a generation probability of the sequence of texture primitives, and the grammar anomaly score is derived by computing a negative log likelihood of the generation probability. This computation process can be represented by the following equation:

[0031]

[0032] wherein:

[0033] represents the grammar anomaly score;

[0034] represents a sequence of texture primitives to be analyzed, which is composed of texture primitives, i.e.

[0035] ​​​​​​ the global texture syntax model judges the texture primitive sequence a generation probability;

[0036] the generation probability a negative log-likelihood value.

[0037] In one specific embodiment, the information fusion module is specifically configured to perform point-by-point operation on the values in the structural residual graph and the syntax anomaly graph to generate the comprehensive anomaly response graph, and the point-by-point operation includes the following steps:

[0038] obtaining a structural residual value and a syntax anomaly score at a same spatial position in the structural residual graph and the syntax anomaly graph;

[0039] applying a first preset fusion weight to the structural residual value to obtain a weighted structural residual value, and applying a second preset fusion weight to the syntax anomaly score to obtain a weighted syntax anomaly score;

[0040] adding the weighted structural residual value and the weighted syntax anomaly score to obtain a response value of the comprehensive anomaly response graph at the same spatial position. The point-by-point operation process can be represented by the following formula:

[0041] ;

[0042] wherein:

[0043] represents a spatial position coordinate in a two-dimensional graph;

[0044] represents a response value of the comprehensive anomaly response graph at the spatial position ;

[0045] represents a structural residual value of the structural residual graph at the spatial position ;

[0046] represents a syntax anomaly score of the syntax anomaly graph at the spatial position ;

[0047] represents the first preset fusion weight;

[0048] represents the second preset fusion weight;

[0049] represents the spatial position a weighted structural residual value at the location;

[0050] a weighted syntax abnormality score at the location a weighted structural residual value at the location.

[0051] Preferably, the defect decision module structurally comprises:

[0052] a defect locating unit connected to the information fusion module and configured to perform threshold segmentation and connected domain analysis on the received comprehensive abnormality response map, so as to locate the defect area;

[0053] and a defect classification unit connected to the defect locating unit and configured to extract preset features of the defect area, and then divide the defect area into types by using a classifier.

[0054] In a specific embodiment, the texture primitive is an image block representing the smallest repeatable weaving structure unit in the fabric image.

[0055] Further, both the local generative model and the global texture syntax model are deep learning models trained by unsupervised learning on normal fabric samples without defects. This training method enables the model to autonomously learn the internal rules of normal fabric without relying on any sample with defect annotation.

[0056] The present application provides a deep learning-based intelligent detection and classification system for knitted fabric defects. It has the following advantages:

[0057] 1. The present application sets up a local verification module and a global verification module to perform parallel analysis on fabric images from two dimensions of local weaving rules and global organization syntax, respectively. The local verification module can identify subtle texture primitive abnormalities, while the global verification module can find large-scale structural errors. Through the information fusion module, the two analysis results are integrated, so that the system can respond to both local and global abnormalities, thereby reducing the false negative rate when facing complex defects with both characteristics.

[0058] 2. In the local verification module, the structural residual value is generated by calculating the multi-dimensional differences between the predicted texture primitive and the target texture primitive in the pixel space, frequency space and gradient space. Compared with single-dimensional difference calculation, this multi-modal residual quantization method can more stably represent the real deviation of the texture, effectively distinguish the normal fluctuations of the fabric from the real defects, and thus reduce the false positive rate of the system.

[0059] 3、The local generation model and the global texture syntax model adopted by the application are both obtained by unsupervised learning on normal fabric samples without defects. This method does not rely on any samples with defect annotations for training, and only needs to learn the internal rules of normal fabric. This eliminates the need to collect and annotate a large number of diverse defect samples, reduces the cost and period of model training, and enables the system to be conveniently applied to different types of fabric detection tasks. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a perspective view of the application;

[0061] Figure 2 is a schematic diagram of the working principle of the local verification module of one embodiment of the application;

[0062] Figure 3 is a schematic diagram of the working principle of the global verification module of one embodiment of the application;

[0063] Figure 4 is a schematic diagram of the working principle of the information fusion module of one embodiment of the application;

[0064] Figure 5 is a workflow diagram of the defect decision module of one embodiment of the application;

[0065] Figure 6 is a schematic diagram of the system model training principle of one embodiment of the application.

[0066] Among them, 10, texture primitive extraction module; 20, local verification module; 30, global verification module; 40, information fusion module; 50, defect decision module. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings of the application. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0068] Referring to the accompanying Figure 1 , Figure 1 is a structural block diagram of a fabric defect intelligent detection and classification system based on deep learning according to one embodiment of the application. The application provides a fabric defect intelligent detection and classification system based on deep learning, which can include: a texture primitive extraction module 10, a local verification module 20, a global verification module 30, an information fusion module 40, and a defect decision module 50.

[0069] In one embodiment of the present application, the workflow of the system starts from the texture primitive extraction module 10. The texture primitive extraction module 10 receives an input fabric image and partitions the fabric image into a two-dimensional primitive matrix composed of a plurality of texture primitives according to a preset size. The two-dimensional primitive matrix is the basic data structure for subsequent processing, which is transmitted to the local verification module 20 and the global verification module 30 simultaneously.

[0070] The local verification module 20 receives the two-dimensional primitive matrix from the texture primitive extraction module 10. For each target texture primitive in the matrix, the local verification module 20 generates a predicted texture primitive according to the context information of the target texture primitive using its configured local generative model. Subsequently, the local verification module 20 generates a structural residual map by calculating the structural residual values between the predicted texture primitive and the target texture primitive at all positions and combining all the residual values. This structural residual map is output to the information fusion module 40.

[0071] Meanwhile, the global verification module 30 also receives the two-dimensional primitive matrix from the texture primitive extraction module 10. The global verification module 30 first converts the two-dimensional primitive matrix into a one-dimensional texture primitive sequence and then analyzes the sequence using its configured global weave grammar model to calculate a grammar anomaly score representing the compliance of the sequence. Based on the score, the global verification module 30 generates a grammar anomaly map. This grammar anomaly map is also output to the information fusion module 40.

[0072] The input end of the information fusion module 40 is connected to the output ends of the local verification module 20 and the global verification module 30, respectively, to receive the structural residual map and the grammar anomaly map. The information fusion module 40 performs point-by-point operations on the two maps to generate a comprehensive anomaly response map. Specifically, at each spatial position in the map, the information fusion module 40 obtains the corresponding structural residual value and grammar anomaly score and calculates the comprehensive anomaly response value by the following formula:

[0073] ;

[0074] Wherein:

[0075] represents a spatial position coordinate in the two-dimensional map;

[0076] represents the response value of the comprehensive anomaly response map at the spatial position ;

[0077] represents the structural residual value of the structural residual map at the spatial position ;

[0078] represents the syntax abnormality score at the spatial position .

[0079] represents the first preset fusion weight;

[0080] represents the second preset fusion weight;

[0081] represents the weighted structural residual value at the spatial position .

[0082] represents the weighted syntax abnormality score at the spatial position .

[0083] Finally, the defect decision module 50 receives the comprehensive abnormality response map generated by the information fusion module 40. The defect decision module 50 first processes according to the response values in the comprehensive abnormality response map to locate the defect area in the fabric image; then, feature extraction and analysis are performed on the located defect area to complete the classification of the defect type. The final output of the system is the position and category information of the defects in the fabric image.

[0084] By detecting the coordinates of the fundamental frequency peak in the frequency domain , the spatial periods of the fabric in the horizontal and vertical directions and can be calculated. The calculation relationship is as follows:

[0085] ;

[0086] ;

[0087] wherein:

[0088] and represent the pixel periods of the fabric in the horizontal and vertical directions, respectively;

[0089] and represent the width and height of the sample image, respectively, in pixels;

[0090] and represent the horizontal and vertical frequency coordinates of the fundamental frequency peak in the frequency spectrum, respectively.

[0091] Based on the calculated spatial periods, the width and height of the texture primitive are set to be equal to or an integer multiple of and The size is chosen to be equal to the fundamental period, which ensures that each texture primitive contains exactly one complete, smallest weaving structure unit. This determined size is used as a key parameter in the subsequent decomposition operation of all input fabric images by the texture primitive extraction module 10. In another embodiment, the periodicity of the sample image can also be determined by calculating its two-dimensional autocorrelation function, the peak position of which also corresponds to the spatial period of the fabric.

[0092] After the size of the texture primitive is determined , the texture primitive extraction module 10 decomposes the received fabric image to be detected according to this size. This decomposition process is realized by a sliding window cutting operation, thereby converting the complete fabric image into a structured two-dimensional primitive matrix.

[0093] In a specific embodiment, the decomposition process starts from the top-left vertex of the fabric image to be detected. An extraction window with a size of is placed at the initial position of the image. The image region located in this window is extracted as the first texture primitive and placed at the first row and first column position of the two-dimensional primitive matrix.

[0094] Subsequently, the extraction window moves on the image with a preset step size. In a preferred embodiment, the horizontal moving step size is set to and the vertical moving step size is set to . This setting ensures that there is no pixel overlap between the texture primitives extracted in two adjacent times, realizing the non-overlapping grid segmentation of the fabric image. The extraction window traverses the entire fabric image in the order from left to right and then from top to bottom, until the right-bottom boundary of the image is reached.

[0095] At each step in the traversal process, the image block covered by the extraction window is extracted as an independent texture primitive. The position index of this texture primitive in the two-dimensional primitive matrix corresponds to the grid coordinates of its corresponding position on the original fabric image. After the entire decomposition process is completed, the original fabric image is converted into a two-dimensional primitive matrix , each element of which is an image block with a size of . This two-dimensional primitive matrix, as the final output of the texture primitive extraction module 10, is simultaneously transmitted to the local verification module 20 and the global verification module 30.

[0096] Referring to the accompanying Figure 2 , the accompanying Figure 2Figure 1 is a schematic diagram showing the working principle of a local verification module according to an embodiment of the present application. The local verification module 20 receives the two-dimensional primitive matrix outputted by the texture primitive extraction module 10. The working principle of the local verification module 20 is based on the contextual dependency of local texture in a fabric structure, i.e. in a defect-free fabric structure, the appearance and structure of any texture primitive is strictly constrained by its neighboring texture primitives.

[0097] The local verification module 20 models this contextual dependency through a locally generated model configured by it. The locally generated model is trained to learn a mapping function whose input is the context of any target texture primitive and whose output is a predicted texture primitive that matches the context. The context is defined as a set of neighboring texture primitives of the target texture primitive in the two-dimensional primitive matrix. In a specific embodiment, for a target texture primitive located at matrix position , its context is the set of texture primitives at its surrounding eight positions , , , , , , , .

[0098] During the defect detection process, the local verification module 20 iterates through each target texture primitive in the two-dimensional primitive matrix. At each position, the local verification module 20 extracts the context of the target texture primitive and inputs the context into the locally generated model. The locally generated model generates a predicted texture primitive that is structurally consistent with the context according to the learned local weaving rules of a normal fabric. The predicted texture primitive represents the ideal appearance that a defect-free texture primitive should have at that specific position.

[0099] Subsequently, the local verification module 20 calculates the structural residual value between the predicted texture primitive and the target texture primitive. The structural residual value is a quantitative indicator that characterizes the degree of inconsistency between the two texture primitives in structure. If the target texture primitive is consistent with the surrounding fabric structure, the difference between it and the predicted texture primitive is small, and the calculated structural residual value is also small. Conversely, if the target texture primitive is part of a defect, its structure deviates from the normal weaving rules, resulting in a significant difference between it and the predicted texture primitive, and the calculated structural residual value also increases. By performing this operation for all positions, a structural residual map that spatially corresponds to the original fabric image is finally generated.

[0100] In a preferred embodiment, the local generation model is a conditional generative adversarial network. The conditional generative adversarial network includes a generator and a discriminator in structure. The role of the generator is to learn the mapping from context to central texture primitive, which receives the context of the target texture primitive as conditional input and outputs the predicted texture primitive. The role of the discriminator is to judge whether the input texture primitive matches its context. In the training phase, the generator and the discriminator are adversarially trained: the generator tries to generate a predicted texture primitive sufficient to make the discriminator misjudge, while the discriminator tries to distinguish between the real target texture primitive and the predicted texture primitive. This training process only uses flawless normal fabric samples, forcing the generator to accurately learn the local weaving rules of normal fabric. In the defect detection phase, only the trained generator is used to generate the predicted texture primitive.

[0101] Referring to the drawings Figure 3 After the local generation model generates a predicted texture primitive according to the context of the target texture primitive, the local verification module 20 then calculates the structural residual value between the predicted texture primitive and the target texture primitive. This calculation process is realized by a multi-modal residual fusion formula, which quantifies and weights the differences between the two texture primitives in multiple feature spaces.

[0102] In a specific embodiment, the calculation formula of the structural residual value is as follows:

[0103] ;

[0104] Wherein:

[0105] represents the final calculated structural residual value;

[0106] represents the representation of the target texture primitive in the pixel space, represents the representation of the predicted texture primitive in the pixel space;

[0107] represents the representation of the target texture primitive in the frequency space, represents the representation of the predicted texture primitive in the frequency space;

[0108] represents the representation of the target texture primitive in the gradient space, represents the representation of the predicted texture primitive in the gradient space;

[0109] , , respectively represent the distance measurement functions for calculating the differences between the two inputs in the pixel space, the frequency space and the gradient space.

[0110] , , represent preset fusion weights of the corresponding pixel space, frequency space and gradient space residuals respectively, and their sum is 1;

[0111] represents a residual value in the pixel space;

[0112] represents a weighted pixel space residual;

[0113] represents a residual value in the frequency space;

[0114] represents a weighted frequency space residual;

[0115] represents a residual value in the gradient space;

[0116] represents a weighted gradient space residual.

[0117] Distance metric function , and The specific calculation process is as follows:

[0118] Pixel space difference Calculation: the absolute value of the difference between the pixel values at the corresponding positions in the two texture primitive image blocks is summed and averaged.

[0119] Frequency space difference Calculation: first, perform two-dimensional discrete Fourier transform on and to obtain and , and then calculate the total sum of the differences in spectral amplitude of all frequency components.

[0120] Gradient space difference Calculation: first, use the Sobel operator to calculate the gradient maps and of and respectively, and then calculate the total sum of the absolute values of the differences between the gradient values at the corresponding positions in the two gradient maps.

[0121] The local verification module 20 performs the aforementioned calculation of the structural residual value for each texture primitive in the two-dimensional primitive matrix received from the texture primitive extraction module 10. This process traverses the entire two-dimensional primitive matrix, performing the calculation for each position. The texture primitive generates a corresponding structural residual value. .

[0122] After completing the calculations at all locations, the local verification module 20 generates a structural residual map. The dimensions of this diagram correspond to the dimensions of the two-dimensional primitive matrix. Each position in the diagram... The value is the structural residual value calculated from the texture primitive at the corresponding position. Therefore, in the structural residual map, the higher the value of a region, the greater the deviation between the corresponding region of the original fabric image and the local weaving pattern of the normal fabric. Ultimately, this structural residual map... As the output of the local verification module 20, it is transmitted to the information fusion module 40.

[0123] See attached document Figure 3 , attached Figure 3 This is a schematic diagram illustrating the working principle of a global verification module according to an embodiment of the present invention. The global verification module 30 is connected to the texture primitive extraction module 10 and is used to receive a two-dimensional primitive matrix. The working principle of the global verification module 30 is to treat the global structure of the fabric as a sequence with specific grammatical rules, and to identify large-scale, structural fabric defects by performing grammatical compliance analysis on the sequence composed of texture primitives.

[0124] In one specific embodiment, the global verification module 30 first converts the received two-dimensional primitive matrix into a one-dimensional texture primitive sequence. This conversion operation is achieved through a preset scanning path, for example, scanning the two-dimensional primitive matrix row by row in the order from left to right and from top to bottom, and connecting each texture primitive in sequence to form a texture primitive sequence that can reflect the global weaving order of the fabric.

[0125] Subsequently, the global verification module 30 analyzes the texture primitive sequence using its configured global texture syntax model. In a preferred embodiment, the global texture syntax model is a context-free grammar model based on a variational autoencoder. Structurally, this model includes an encoder and a decoder. The encoder is responsible for compressing the input texture primitive sequence and mapping it to a low-dimensional, continuous latent probability space, generating a latent representation vector. The decoder is responsible for reconstructing the original texture primitive sequence from the latent representation vector.

[0126] The encoding and decoding process constitutes a probabilistic modeling of the texture primitive sequence. The global texture grammar model is trained using only flawless normal fabric samples, and the training process is a kind of unsupervised learning. Through the training, the model learns the intrinsic probability distribution of the normal fabric texture primitive sequence. In this model, the encoder learns to map the grammar-compliant texture primitive sequence to a compact, continuous region in the latent space; the decoder learns to reconstruct the grammar-compliant texture primitive sequence from any point in the region. Therefore, the distribution of the latent space represents the "context-free grammar" rule set of the normal fabric weave, and the reconstruction process of the input sequence is equivalent to generating and checking the sequence using the implicitly learned grammar rules.

[0127] When a texture primitive sequence that complies with the normal weaving grammar is input, the model can successfully reconstruct it with a low reconstruction error. Conversely, when a sequence containing defects that does not comply with the normal weaving grammar is input, its representation in the latent space will deviate from the normal distribution, causing the decoder to fail to accurately reconstruct, resulting in a higher reconstruction error. The size of the reconstruction error is directly related to the generation probability of the sequence, and is used to calculate the final grammar anomaly score.

[0128] After the global texture grammar model completes the unsupervised learning of normal fabric samples, the global verification module 30 uses the model to analyze the texture primitive sequence converted from the fabric image to be detected, to calculate the grammar anomaly score. The score is used to quantify the degree of deviation of the texture primitive sequence from the learned normal fabric global weave grammar.

[0129] In a specific embodiment, the calculation of the grammar anomaly score is achieved by evaluating the generation probability of the texture primitive sequence. The generation probability is output by the global texture grammar model (context-free grammar model based on variational autoencoder), which reflects the likelihood of the input sequence complying with the normal fabric weave rules. The grammar anomaly score is obtained by calculating the negative logarithm likelihood of the generation probability, and the specific calculation formula is as follows:

[0130] ;

[0131] wherein:

[0132] represents the grammar anomaly score;

[0133] represents the texture primitive sequence to be analyzed composed of texture primitives, i.e. ;

[0134] represents that the global texture grammar model judges the texture primitive sequence The probability of generation. A sequence that matches the normal fabric grammar well will get a higher probability of generation, while a sequence that contains defects or violates the grammar will get a lower probability of generation.

[0135] The negative log-likelihood value of the probability of generation.

[0136] In order to locate the analysis results of the grammar anomaly to the specific spatial position, the global verification module 30 traverses the one-dimensional texture primitive sequence in a sliding window manner. A window with a preset length slides on the sequence position by position, and at each position, the sub-sequence covered by the window is input into the global texture grammar model as an independent unit, and a corresponding grammar anomaly score is calculated according to the above formula.

[0137] After completing the sliding window calculation on the entire texture primitive sequence, the global verification module 30 will generate a grammar anomaly map The generation process of the map includes: creating a two-dimensional grid with the same dimension as the two-dimensional primitive matrix for the grammar anomaly map; then, assigning the grammar anomaly score calculated at each sliding window position to the coordinate position in the two-dimensional grid corresponding to the center texture primitive of the sliding window .

[0138] In this way, a single scalar value representing the degree of global sequence anomaly is converted into a two-dimensional grammar anomaly map that can indicate the specific position of the anomaly. The higher the value in the grammar anomaly map, the greater the possibility that the corresponding area of the original fabric image has a grammar error in the global organization structure. Finally, the grammar anomaly map is output by the global verification module 30 and transmitted to the information fusion module 40.

[0139] Referring to the accompanying Figure 4 , the accompanying Figure 4 is a schematic diagram of the working principle of the information fusion module according to an embodiment of the present application. The information fusion module 40 is connected to the output terminals of the local verification module 20 and the global verification module 30, respectively, and its function is to receive the structural residual map generated by the local verification module 20 and the grammar anomaly map generated by the global verification module 30, and combine the defect response information contained in the two.

[0140] The structural residual map mainly represents the structural anomaly of the fabric at the micro-texture primitive level, while the grammar anomaly map represents the grammar error of the fabric at the macro-organization sequence level. By fusing these two types of information, the system can simultaneously have the response ability to local subtle defects and global structural defects, so as to effectively respond to various types of defects.

[0141] ​In one embodiment, the information fusion is achieved by point-by-point weighted operation on the two input maps. Since both the structural residual map and the syntax abnormality map are generated based on the original two-dimensional primitive matrix, they are consistent in dimension, and there is a one-to-one correspondence between the coordinates of any position in the two maps. The information fusion module 40 traverses all spatial positions in the map and performs the following fusion calculation for each position:

[0142] ;

[0143] wherein:

[0144] represents a spatial position coordinate in the two-dimensional map;

[0145] represents the response value of the comprehensive abnormality response map at the spatial position ;

[0146] represents the structural residual value of the structural residual map at the spatial position ;

[0147] represents the syntax abnormality score of the syntax abnormality map at the spatial position ;

[0148] represents the first preset fusion weight;

[0149] represents the second preset fusion weight;

[0150] represents the weighted structural residual value at the spatial position ;

[0151] represents the weighted syntax abnormality score at the spatial position .

[0152] The first preset fusion weight and the second preset fusion weight are non-negative constants, and in one embodiment, their sum is 1, i.e. The specific values of the two weights can be set according to the characteristics of the fabric to be detected and the main defect types. For example, for fabrics prone to small and isolated defects, the value of can be increased to focus on local verification results; for fabrics prone to large-area and regular damage defects, the value of can be increased to focus on global verification results.

[0153] In another embodiment, to obtain the optimal fusion effect, the weights and The value can be determined by performing a grid search on a small-scale validation dataset containing a small number of known defect samples. By traversing multiple different weight combinations and using evaluation metrics such as the F1 score for defect detection as the objective function, the weight combination that maximizes the objective function value is finally selected as the final configuration parameter of the system. Specifically, the grid search execution steps are: setting weights... The search range is 0 to 1, with a step size of 0.1; for each Value, corresponding settings Use each group The weighted combination performs information fusion and defect decision-making on all images in the validation dataset and calculates their F1 scores; finally, the weights that maximize the F1 score are selected. The combination serves as the system's solidification parameter.

[0154] This point-by-point calculation process traverses all spatial locations. Finally, a comprehensive anomaly response map with the same size as the input map is generated. The values ​​in this graph integrate information from both local structural deviations and global syntax errors. Higher values ​​indicate a greater probability that the region corresponds to a defect in the original fabric image. This integrated anomaly response graph, as the final output of the information fusion module 40, is transmitted to the defect decision module 50 as the direct basis for subsequent defect localization and classification.

[0155] See attached document Figure 5 , attached Figure 5 This is a flowchart of a defect decision-making module according to an embodiment of the present invention. The defect decision-making module 50 is connected to the output of the information fusion module 40, and receives the comprehensive anomaly response diagram generated by the information fusion module 40. It is responsible for the final location and classification of defective areas. In an embodiment of the present invention, the defect decision module 50 structurally includes a defect location unit and a defect classification unit.

[0156] The defect localization unit first performs a threshold segmentation operation on the input composite anomaly response map. The purpose of this operation is to separate potentially defective regions (foreground) with higher numerical values ​​from normal background regions (background) in the image, thereby generating a binary image. In one specific embodiment, the threshold used for segmentation... The threshold is automatically determined using the Otsu method. This method adaptively finds the optimal segmentation threshold by calculating a gray level that maximizes the inter-class variance between the foreground and background pixel classes, thereby reducing the reliance on manually set thresholds.

[0157] After generating the binary image, the defect locating unit performs a connected component analysis on it. This analysis process scans the binary image and combines spatially contiguous foreground pixels into independent connected regions. Each independent connected region is considered as a candidate defect region. In this process, a morphological opening operation can be performed first to remove isolated foreground points with too small area caused by random noise, and a morphological closing operation can be performed to fill small holes that can exist inside the candidate defect regions. After completing the connected component analysis, the defect locating unit calculates the minimum bounding box, i.e. the bounding box, for each of the finally determined defect regions. The coordinate and size information of the bounding box constitutes the final positioning result of the defect location.

[0158] The defect classification unit receives the defect region information output by the defect locating unit. For each of the located defect regions, the defect classification unit first extracts the corresponding image patch from the original input fabric image according to its bounding box. Then, a set of pre-defined numerical features that describe the characteristics of the defect image patch are extracted from it, and these features are combined into a feature vector. In one embodiment, the extracted features include:

[0159] Geometric features: area (total number of pixels), perimeter, aspect ratio, compactness of the region.

[0160] Texture features: contrast, correlation, energy and homogeneity calculated based on the gray level co-occurrence matrix (GLCM).

[0161] After extracting the feature vectors of all defect regions, the defect classification unit inputs each feature vector into a pre-trained classifier. In one embodiment, the classifier is a support vector machine. The support vector machine is trained through an independent supervised learning process, and its training set contains feature vectors extracted from a large number of samples of known types of defects (e.g. broken warp, broken weft, oil stains, etc.) and their corresponding class labels. In another preferred embodiment, the function of the defect classification unit is realized by a pre-trained convolutional neural network (CNN). Specifically, the defect image patch extracted from the original fabric image is input into a convolutional neural network (e.g. ResNet or MobileNet) that is pre-trained on a large general-purpose image dataset (e.g. ImageNet) and fine-tuned. The network directly outputs the class label corresponding to the defect image patch, thereby integrating the feature extraction and classification steps into a unified deep learning model. The classifier outputs a specific defect class label according to the input feature vector.

[0162] Finally, the defect decision module 50 integrates the bounding box information output by the defect localization unit and the category label output by the defect classification unit to form the final detection result. The result clearly indicates the precise position, size and category of each detected defect in the fabric image.

[0163] Referring to the drawings Figure 6 , Figure 6 is a schematic diagram of the system model training principle according to an embodiment of the present application. In the embodiment of the present application, the core analysis models of the system, i.e. the local generation model configured in the local verification module 20 and the global texture grammar model configured in the global verification module 30, are both trained through a non-supervised learning paradigm. This training paradigm determines the data preparation process of the system before deployment, which does not rely on any fabric samples containing defects.

[0164] The construction of the training data set starts with the collection of multiple batches of normal fabric images confirmed not to contain any weaving defects. In an embodiment, the collected image samples cover the slight fluctuations of the same type of fabric in the normal production process, such as slight unevenness of yarn thickness or slight fluctuation of weaving density. This is done to enable the model to learn the inherent variation range of normal fabric to distinguish between normal fluctuations and real defects.

[0165] All collected normal fabric image samples are input to the texture primitive extraction module 10. The texture primitive extraction module 10 performs a decomposition operation on each normal fabric image to convert it into a corresponding two-dimensional primitive matrix. These two-dimensional primitive matrices generated from normal fabric images constitute the basic data source required for training.

[0166] For the training of the local generation model, training sample pairs are extracted from the basic data source. Specifically, all texture primitives in each two-dimensional primitive matrix are traversed, and for each texture primitive, it is taken as the target output and its neighboring texture primitive set in the matrix is taken as the conditional input. In this way, a large number of sample pairs composed of (context, center texture primitive) are constructed, which serve as the training data set of the local generation model (conditional generative adversarial network).

[0167] For the training of the global texture grammar model, training samples are also extracted from the basic data source. Specifically, each two-dimensional primitive matrix is converted into a one-dimensional texture primitive sequence through a preset scanning path (e.g. row-by-row scanning). All texture primitive sequences generated from normal fabric samples collectively constitute the training data set of the global texture grammar model (context-free grammar model based on variational autoencoder). The goal of this training process is to enable the model to learn the probability distribution of these normal sequences.

[0168] For the local generative model (conditional generative adversarial network) configured in the local verification module 20, its training process is an adversarial learning process, which specifically includes the following steps. The generator and the discriminator of the model update their respective network parameters alternately in the training process. In each step of the training, first, the parameters of the generator are fixed, and the discriminator is trained. The real (context, central texture primitive) sample pair extracted from the normal fabric sample is input to the discriminator, and its category label is set to "true"; at the same time, the real context is input to the generator to generate a predicted texture primitive, and the (context, predicted texture primitive) sample pair is input to the discriminator, and its category label is set to "false". The parameters of the discriminator are updated by calculating the classification loss, so that the ability of the discriminator to distinguish between true and false sample pairs is optimized.

[0169] Subsequently, the parameters of the discriminator are fixed, and the generator is trained. The real context is input to the generator to generate a predicted texture primitive, and the (context, predicted texture primitive) sample pair is input to the updated discriminator. At this time, the goal of the generator is to make the sample pair it generates be judged as "true" by the discriminator. Therefore, the parameters of the generator are updated by a loss function aimed at deceiving the discriminator. In addition, in order to ensure that the predicted texture primitive output by the generator is similar in structure to the real texture primitive, a reconstruction loss term is also included in the loss function of the generator, which calculates the L1 norm distance between the predicted texture primitive and the real central texture primitive. The whole training process is iterated until the generator can generate normal texture primitives from the context that cannot be effectively distinguished by the discriminator and have high fidelity, at which time the model training is completed.

[0170] For the global texture grammar model (context-free grammar model based on variational autoencoder) configured in the global verification module 30, its training process aims to minimize a composite loss function consisting of two components. The training process selects a sequence consisting of normal texture primitives from the training data set and inputs it to the encoder of the model. The encoder maps the input sequence to the parameters of a probability distribution in the latent space, i.e. the mean vector and the variance vector.

[0171] A latent representation vector is sampled from the latent space distribution and input to the decoder of the model. The decoder reconstructs the original texture primitive sequence from the latent representation vector. The first loss function component in the training process is the reconstruction loss, which quantifies the reconstruction accuracy of the model by calculating the mean square error between the original input sequence and the decoder reconstructed sequence.

[0172] The second component of the compound loss function is the KL divergence. This term computes the difference between the latent space probability distribution output by the encoder and a standard normal distribution. This term serves as a regularization term to ensure that the latent space learned by the model has a regular, continuous structure. The total loss of the model is the weighted sum of the reconstruction loss and the KL divergence. The network parameters of both the encoder and the decoder are updated simultaneously according to the total loss by the backpropagation algorithm. The training process iterates until the total loss of the model on the normal texture primitive sequence converges to a stable small value, indicating that the model has sufficiently learned the intrinsic probability distribution of the normal fabric sequence.

Claims

1. A deep learning-based intelligent detection and classification system for defects in knitted fabric, characterized in that, The method comprises the following steps: a texture primitive extraction module is configured to decompose an input fabric image into a two-dimensional primitive matrix composed of a plurality of texture primitives; a local verification module is configured with a local generative model, which is a deep learning model trained via normal fabric samples to learn fabric local weaving rules, and the local verification module uses the local generative model to generate a predicted texture primitive according to the context of a target texture primitive, and calculates a structural residual value between the predicted texture primitive and the target texture primitive, thereby generating a structural residual map; a global verification module is configured with a global texture grammar model, which is a deep learning model trained via normal fabric samples to learn fabric global organization grammar, and the global verification module uses the global texture grammar model to analyze a texture primitive sequence converted from the two-dimensional primitive matrix, thereby calculating a grammar anomaly score and generating a grammar anomaly map; wherein the global texture grammar model is a context-free grammar model based on a variational autoencoder, and the context-free grammar model based on the variational autoencoder comprises an encoder and a decoder, which are used to calculate the grammar anomaly score by probabilistic modeling of the texture primitive sequence; the probabilistic modeling is used to determine the generation probability of the texture primitive sequence, and the grammar anomaly score is calculated by taking the negative logarithm likelihood of the generation probability; an information fusion module is configured to fuse the structural residual map and the grammar anomaly map, thereby generating a comprehensive anomaly response map; a defect decision module is configured to locate and classify defect regions in the fabric image according to the comprehensive anomaly response map.

2. The deep learning based knitted fabric defect intelligent detection and classification system according to claim 1, wherein, The local generative model is a conditional generative adversarial network, which comprises a generator configured to receive the context of the target texture primitive as input to generate the predicted texture primitive.

3. The deep learning based knitted fabric defect intelligent detection and classification system according to claim 1, wherein, The structural residual value is calculated based on the difference between the predicted texture primitive and the target texture primitive in at least two or more feature spaces: pixel space, frequency space, and gradient space.

4. The deep learning based knitted fabric defect intelligent detection and classification system according to claim 1, wherein, The information fusion module is specifically configured to perform point-by-point operation on the values in the structural residual map and the grammar anomaly map to generate the comprehensive anomaly response map, and the point-by-point operation step comprises: obtaining a structural residual value and a grammar anomaly score at the same spatial position in the structural residual map and the grammar anomaly map; applying a first preset fusion weight to the structural residual value to obtain a weighted structural residual value, and applying a second preset fusion weight to the grammar anomaly score to obtain a weighted grammar anomaly score; adding the weighted structural residual value and the weighted grammar anomaly score to obtain a response value of the comprehensive anomaly response map at the same spatial position.

5. The deep learning based knitted fabric intelligent defect detection and classification system according to claim 1, wherein, The defect decision module comprises: a defect locating unit connected to the information fusion module and configured to perform threshold segmentation and connected domain analysis on the received comprehensive anomaly response map to locate the defect region. A defect classification unit is connected to the defect locating unit and is configured to extract preset features of the defect area and use a classifier to classify the defect area.

6. The deep learning based knitted fabric intelligent defect detection and classification system according to claim 1, wherein, The texture primitive is an image block representing the smallest repeatable weaving structure unit in the fabric image.

7. The deep learning based knitted fabric defect intelligent detection and classification system according to claim 3, wherein, The structural residual value is calculated by a multi-modal residual fusion formula which is used to quantify and weightedly sum the differences in the pixel space, the frequency space and the gradient space.

8. The deep learning based knitted fabric intelligent defect detection and classification system according to claim 1, wherein, Both the local generative model and the global texture syntax model are deep learning models trained by unsupervised learning on normal fabric samples without defects.

Images