Method and system for detecting state of napping woven fabric based on machine vision

Abstract

The invention relates to the technical field of image recognition, in particular to a napping woven fabric state detection method and system based on machine vision. The method comprises the following steps: firstly, acquiring a napping woven fabric image, and labeling a state label to obtain a first data set; then designing different illumination conditions, constructing an image illumination simulation model, generating an illumination simulation image according to the designed illumination conditions and the napping woven fabric image, and generating illumination simulation images for all the images in the first data set to obtain a second data set; constructing a state detection model, training the state detection model through the second data set, obtaining a to-be-detected napping woven fabric image, and generating a to-be-detected illumination simulation image according to a designed illumination condition; and finally, inputting a to-be-detected illumination simulation image and a to-be-detected napping woven fabric image into the trained state detection model to obtain a napping woven fabric state detection result. According to the invention, the accuracy and robustness of napping woven fabric state detection in a single light environment can be improved.

Description

Technical Field

[0001] The invention relates to the technical field of image recognition, and in particular to a method and system for detecting the state of napped woven fabrics based on machine vision. Background Art

[0002] Fabric napping machine is a kind of equipment widely used in textile industry, mainly used to treat the surface of textiles and improve the feel and appearance quality of fabrics. The napping machine treats the surface of fabrics by stretching, rubbing and cutting, etc., to produce a hairiness effect, making the surface of fabrics fluffy. With the continuous development of textile processing technology, especially the increasing demand for high-end fabric products, the surface fluff state of fabrics has become a key factor in product quality control.

[0003] With the rapid development of computer vision technology, the state detection method of napped woven fabrics based on machine vision has solved the problems of slow response speed, insufficient precision and poor adaptability of traditional methods. The machine vision system can efficiently and accurately obtain various information on the fabric surface through high-speed cameras, image processing technology and deep learning algorithms, improve the automation, precision and speed of fabric surface state detection, and make machine vision technology play an increasingly important role in the real-time monitoring and analysis of the state of napped woven fabrics. However, there are still some problems with the existing state detection method of napped woven fabrics based on machine vision. First, different lighting conditions will change the visual performance of napped fibers. Under a single lighting environment, it is impossible to detect more comprehensive details of the napped state, and it is impossible to improve the accuracy of the state detection of napped woven fabrics; and for different lighting conditions, there is a lack of sufficient napped woven fabric image data basis, which makes the state detection of napped woven fabrics lack of robustness.

[0004] Therefore, a method and system for detecting the state of napped woven fabric based on machine vision are proposed. Summary of the invention

[0005] The object of the present invention is to provide a method and system for detecting the state of a napped woven fabric based on machine vision. First, a first data set is obtained by acquiring an image of the napped woven fabric and marking a state label; then different lighting conditions are designed, and an image lighting simulation model is constructed. According to the designed lighting conditions and the image of the napped woven fabric, a lighting simulation image is generated, and a lighting simulation image is generated for all images in the first data set to obtain a second data set; then a state detection model is constructed, and the state detection model is trained with the second data set, and an image of the napped woven fabric to be detected is acquired, and a lighting simulation image to be detected is generated according to the designed lighting conditions; finally, the lighting simulation image to be detected and the image of the napped woven fabric to be detected are input into the trained state detection model to obtain a detection result of the napped woven fabric state.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for detecting the state of a napped woven fabric based on machine vision, comprising:

[0008] Designing a first lighting condition, a second lighting condition, and a third lighting condition by means of lighting parameters; the lighting parameters include light source type, color temperature, illumination, and incident angle;

[0009] Obtain N images of napped woven fabrics and annotate them with status labels to obtain a first data set; the status labels include fabric defect labels and fabric status score labels; the fabric defect labels include defect locations and defect categories;

[0010] Constructing an image lighting simulation model, inputting the first lighting condition, the second lighting condition, the third lighting condition and the napped woven fabric image into the image lighting simulation model, generating a first lighting simulation image, a second lighting simulation image and a third lighting simulation image, and traversing all the napped woven fabric images in the first data set to obtain a second data set of 4N in number;

[0011] Building a state detection model, and training the state detection model using the second data set;

[0012] Acquire the image of the napped woven fabric to be detected, input the first lighting condition, the second lighting condition, the third lighting condition and the image of the napped woven fabric to be detected into the image lighting simulation model, and generate a first lighting simulation image to be detected, a second lighting simulation image to be detected and a third lighting simulation image to be detected;

[0013] The first illumination simulation image to be detected, the second illumination simulation image to be detected, the third illumination simulation image to be detected and the napped woven fabric image to be detected are input into the trained state detection model to obtain the napped woven fabric state detection result.

[0014] Preferably, the image illumination simulation model comprises a first input processing layer, a downsampling layer, an upsampling layer and an output processing layer; the first input processing layer performs pixel value normalization processing on the input image to obtain a first feature map, and maps the illumination conditions to d embed dimensional space to obtain an illumination condition vector; the downsampling layer includes n downsampling blocks, through which the first feature map is downsampled, multi-level features are extracted, and a second feature map is obtained; the upsampling layer includes n upsampling blocks and n illumination condition injection blocks, through which the second feature map is upsampled, and the illumination condition vector is fused with the output of the upsampling block through the illumination condition injection block to obtain a third feature map; the output processing layer performs convolution and denormalization on the third feature map to generate an illumination simulation image.

[0015] Preferably, the image illumination simulation model further includes a skip connection mechanism, and the skip connection mechanism transmits the output of the down-sampling block to the up-sampling block of the corresponding level.

[0016] Preferably, the lighting condition injection block includes:

[0017] The output of the i-th upsampling block is reshaped to obtain a feature map vector, which is expressed as:

[0018]

[0019] Among them, F i Represents the feature map output by the i-th upsampling block; Indicates F i The reshaped feature map vector; C i Indicates F i The number of channels; H i and W i Indicates F i The height and width of ; R represents a real number;

[0020] The conditional attention weight is calculated according to the feature map vector and the illumination condition vector, and the calculation formula is:

[0021]

[0022] Among them, a i represents the conditional attention weight of the i-th illumination condition injection block; softmax represents a normalization function; express The transpose of W e A projection matrix representing the illumination condition vector e;

[0023] The conditional attention weight is multiplied by the feature map vector and the feature map is reshaped to obtain the third feature map.

[0024] Preferably, the image illumination simulation model loss includes reconstruction loss, perception loss and adversarial loss, and the calculation formula is as follows:

[0025]

[0026] L adv = -E[log(D(I gen ))];

[0027] Among them, L rec represents the reconstruction loss; N p Indicates the number of pixels of the input image; Represents the real image I gtThe value corresponding to the i-th pixel in ; represents the image I generated by the image illumination simulation model gen The value corresponding to the i-th pixel in L perc represents the perceived loss; N′ p Indicates I gt The number of pixels in the feature map extracted by the pre-trained convolutional neural network; Indicates I gt The value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; Indicates I gen The value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; L adv Denotes the adversarial loss; D(I gen ) indicates I gen The value obtained after being processed by the discriminator; E[·] represents the expected value of all training samples.

[0028] Preferably, the discriminator is composed of a convolutional layer and a fully connected layer; after updating the image illumination simulation model parameters m times, the discriminator parameters are updated once; the image illumination simulation model parameters are updated by calculating the image illumination simulation model loss, and the image illumination simulation model loss calculation formula is:

[0029]

[0030] in, represents the image illumination simulation model loss; Represents the average reconstruction loss of all training samples; represents the average perceptual loss of all training samples; λ1 and λ2 represent and L adv The weight coefficient of

[0031] The discriminator parameters are updated by calculating the discriminator loss, and the discriminator loss calculation formula is:

[0032] L D = -E[log(D(I gt ))+log(1-D(I gen ))];

[0033] Among them, D(I gt ) represents the real image I gt The value obtained after being processed by the discriminator.

[0034] Preferably, the state detection model includes a second input processing layer, a feature extraction layer, a feature fusion layer and a dual-branch output layer;

[0035] The second input processing layer normalizes the first illumination simulation image to be detected, the second illumination simulation image to be detected, the third illumination simulation image to be detected, and the napped woven fabric image to be detected and splices them in terms of the number of channels to obtain an initial comprehensive feature map;

[0036] The feature extraction layer adopts a ResNet-50 structure, introduces a channel attention mechanism in each residual block of ResNet-50, and extracts first comprehensive feature maps of different scales according to the initial comprehensive feature map;

[0037] The feature fusion layer adopts a multi-scale feature pyramid structure, and fuses features of different levels according to the first comprehensive feature map to obtain a second comprehensive feature map of different scales;

[0038] The dual-branch output layer obtains a state detection result according to the second comprehensive feature map of different scales; the dual-branch output layer includes a defect detection branch and a state scoring branch; the state detection result includes a defect detection result vector and a state score; the defect detection branch outputs the defect detection result vector; the state scoring branch outputs the state score.

[0039] Preferably, the channel attention mechanism includes:

[0040] Generate a global feature vector for each channel by global average pooling of the input feature map;

[0041] Generate channel attention weights through a multilayer perceptron according to the global feature vector;

[0042] Multiplying the channel attention weight by the input feature map of the channel attention mechanism element by element in the channel dimension to generate a channel weighted feature map;

[0043] The channel weighted feature map is added to the input feature map of the residual block through a skip connection mechanism to obtain an output feature map of each residual block.

[0044] Preferably, the state detection model loss includes defect detection loss and state scoring loss, and the calculation formula is as follows:

[0045]

[0046] Among them, L det represents the defect detection loss; L focal represents the defect classification loss calculated by focal loss; L giou represents the defect localization loss calculated by the intersection-over-union loss; and represents the weight coefficient; represents the state detection model loss; L scoreDenotes the state score loss.

[0047] A machine vision-based napped woven fabric state detection system, comprising:

[0048] A lighting simulation design module, which designs a first lighting condition, a second lighting condition and a third lighting condition through lighting parameters;

[0049] A data set acquisition module acquires N images of napped woven fabrics and annotates them with state labels to obtain a first data set;

[0050] A simulation image generation module is used to construct an image illumination simulation model, input the first illumination condition, the second illumination condition, the third illumination condition and the napped woven fabric image into the image illumination simulation model, generate a first illumination simulation image, a second illumination simulation image and a third illumination simulation image, and traverse all the napped woven fabric images in the first data set to obtain a second data set of 4N in number;

[0051] A state detection design module is used to construct a state detection model and train the state detection model using the second data set;

[0052] The real-time state detection module obtains the image of the napped woven fabric to be detected, inputs the first lighting condition, the second lighting condition, the third lighting condition and the image of the napped woven fabric to be detected into the image lighting simulation model, generates the first lighting simulation image to be detected, the second lighting simulation image to be detected and the third lighting simulation image to be detected; inputs the first lighting simulation image to be detected, the second lighting simulation image to be detected, the third lighting simulation image to be detected and the image of the napped woven fabric to be detected into the trained state detection model, and obtains the state detection result of the napped woven fabric.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. The present invention designs a variety of lighting conditions for lighting simulation of napped woven fabric images, combines the lighting conditions with the napped woven fabric images through an image lighting simulation model, generates corresponding lighting simulation images, fully considers the influence of lighting conditions on fabric state detection results, and generates a training data set with higher robustness through diversified lighting simulations, thereby enhancing the generalization ability of the state detection model; the image lighting simulation model adopts a combination of up and down sampling layers and lighting condition injection blocks, and is supplemented by a jump connection mechanism to ensure that the simulated image can accurately restore the detailed features under different lighting conditions; it not only effectively solves the problem of insufficient image training data under a single lighting condition, but also improves the adaptability of the state detection model in complex lighting scenes, thereby significantly improving the accuracy and robustness of fabric state detection.

[0055] 2. The state detection model in the present invention adopts a dual-branch structure, including a defect detection branch and a state scoring branch, which realizes the synchronous processing of the defect detection and state scoring of the napped woven fabric; the model introduces a channel attention mechanism in the feature extraction layer, so that the feature extraction pays more attention to the key channel information; the feature fusion layer can fuse features from different scale levels to ensure that the detection results can accurately identify defects of different sizes and complexities, and realize the efficient joint processing of fabric defect detection and state scoring; ensure the consistency of defect detection and state evaluation, and further improve the accuracy of the state detection of the napped woven fabric.

[0056] 3. The illumination simulation model ensures that the generated image maintains the illumination characteristics and has a sense of reality through comprehensive optimization of reconstruction loss, perception loss and adversarial loss. The state detection model combines defect detection loss and state scoring loss. The defect detection loss adopts a method combining focal loss and intersection-over-union loss to improve the detection capability and positioning accuracy of small defects. The state scoring loss uses weighted square error and threshold control to make the scoring more stable and reasonable, effectively balancing the accuracy and stability of the model, and further improving the model's detection effect on the state of napped woven fabrics. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic flow chart of a method for detecting the state of napped woven fabrics based on machine vision provided by an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of a jump connection mechanism of an image illumination simulation model provided by an embodiment of the present invention;

[0059] Figure 3 A schematic structural diagram of a machine vision-based napped woven fabric state detection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Embodiment 1

[0062] During the state detection of napped woven fabrics, due to the particularity of the napped woven fabric surface, the manifestation of some defects will change significantly with the change of lighting conditions. For example, the abnormal direction of the napped woven fabrics that is difficult to observe under direct lighting will be very obvious under oblique lighting; and some surface defects are easier to identify under high color temperature light sources, but may be ignored under low color temperature light sources. The present invention provides a method for detecting the state of napped woven fabrics based on machine vision, which can simulate the images of napped woven fabrics under different lighting conditions, improve the accuracy and robustness of state detection, such as Figure 1 As shown, including:

[0063] Designing a first lighting condition, a second lighting condition, and a third lighting condition by means of lighting parameters; the lighting parameters include light source type, color temperature, illumination, and incident angle;

[0064] Obtain N images of napped woven fabrics and annotate them with status labels to obtain a first data set; the status labels include fabric defect labels and fabric status score labels; the fabric defect labels include defect locations and defect categories;

[0065] Constructing an image lighting simulation model, inputting the first lighting condition, the second lighting condition, the third lighting condition and the napped woven fabric image into the image lighting simulation model, generating a first lighting simulation image, a second lighting simulation image and a third lighting simulation image, and traversing all the napped woven fabric images in the first data set to obtain a second data set of 4N in number;

[0066] Building a state detection model, and training the state detection model using the second data set;

[0067] Acquire the image of the napped woven fabric to be detected, input the first lighting condition, the second lighting condition, the third lighting condition and the image of the napped woven fabric to be detected into the image lighting simulation model, and generate a first lighting simulation image to be detected, a second lighting simulation image to be detected and a third lighting simulation image to be detected;

[0068] The first illumination simulation image to be detected, the second illumination simulation image to be detected, the third illumination simulation image to be detected and the napped woven fabric image to be detected are input into the trained state detection model to obtain the napped woven fabric state detection result.

[0069] The present application embodiment selects three most representative lighting conditions, as shown in Table 1, including: an LED light source with a 45-degree incident angle, which is used to highlight the direction of the fluff and the concave-convex defects on the fabric surface; a fluorescent light source with a 90-degree direct incident angle, which is used to observe the uniformity and stain defects on the fabric surface; and a halogen light source with a 30-degree low angle, which is used to enhance the shadow contrast of the subtle defects on the fabric surface. The combination of these three lighting conditions can fully reflect various characteristics of the fabric surface.

[0070] Table 1 Lighting parameters under different lighting conditions

[0071] Lighting conditions Light source type Color temperature (K) Illuminance(Lux) Angle of incidence (°) First lighting condition led 5500 800 45 Second lighting condition Fluorescent lamp 4000 500 90 The third lighting condition Halogen lamp 3000 300 30

[0072] Furthermore, the image illumination simulation model includes a first input processing layer, a downsampling layer, an upsampling layer and an output processing layer; the first input processing layer performs pixel value normalization processing on the input image to obtain a first feature map, and maps the illumination conditions to d embed dimensional space to obtain an illumination condition vector; the downsampling layer includes n downsampling blocks, through which the first feature map is downsampled, multi-level features are extracted, and a second feature map is obtained; the upsampling layer includes n upsampling blocks and n illumination condition injection blocks, through which the second feature map is upsampled, and the illumination condition vector is fused with the output of the upsampling block through the illumination condition injection block to obtain a third feature map; the output processing layer performs convolution and denormalization on the third feature map to generate an illumination simulation image.

[0073] The first input processing layer ensures the uniformity of input data and the effective expression of lighting parameters through pixel value normalization and lighting condition mapping; the multi-level downsampling-upsampling structure can extract image features of different scales, which helps to understand the visual characteristics of fabrics more comprehensively; the lighting condition injection block fuses the lighting parameter information with the image features to achieve precise control of the lighting effect.

[0074] In an embodiment of the present application, the downsampling layer includes 5 downsampling blocks, each of which is downsampled by convolution and maximum pooling; the upsampling layer includes 5 upsampling blocks and 5 illumination condition injection blocks, and each upsampling block is upsampled by deconvolution and convolution.

[0075] Furthermore, if Figure 2 As shown, the image illumination simulation model also includes a skip connection mechanism, which transmits the output of the downsampling block to the upsampling block of the corresponding level.

[0076] A skip connection mechanism is added to the lighting simulation model to transfer the output of the downsampling block to the corresponding upsampling block, which effectively alleviates the problem of feature loss during the upsampling and downsampling process, ensures the detail integrity of the simulated image, improves the realism of the lighting simulation image, and makes the model more accurate in detecting fabrics.

[0077] Furthermore, the lighting condition injection block includes:

[0078] The output of the i-th upsampling block is reshaped to obtain a feature map vector, which is expressed as:

[0079]

[0080] Among them, F i Represents the feature map output by the i-th upsampling block; Indicates F i The reshaped feature map vector; C i Indicates F i The number of channels; H i and W i Indicates F i The height and width of ; R represents a real number;

[0081] The conditional attention weight is calculated according to the feature map vector and the illumination condition vector, and the calculation formula is:

[0082]

[0083] Among them, a i represents the conditional attention weight of the i-th illumination condition injection block; softmax represents a normalization function; express The transpose of W e A projection matrix representing the illumination condition vector e;

[0084] The conditional attention weight is multiplied by the feature map vector and the feature map is reshaped to obtain the third feature map. The calculation formula is:

[0085]

[0086] Wherein, F3 represents the third feature map; * represents element-by-element multiplication; reshape represents a reshaping operation; Represents the feature map F output by the nth upsampling block n The reshaped feature map vector; a n represents the conditional attention weight of the nth illumination condition injection block; C n Indicates F n The number of channels; H n and Wn Indicates F n height and width.

[0087] The lighting condition injection block generates the third feature map by calculating and fusing the conditional attention weights, which enhances the fusion effect between the lighting conditions and the feature map and makes the lighting simulation more realistic. The introduction of conditional attention weights optimizes the utilization of feature information and improves the quality of simulated images.

[0088] Furthermore, the image illumination simulation model loss includes reconstruction loss, perception loss and adversarial loss, and the calculation formula is as follows:

[0089]

[0090] L adv = -E[log(D(I gen ))];

[0091] Among them, L rec represents the reconstruction loss; N p Indicates the number of pixels of the input image; Represents the real image I gt The value corresponding to the i-th pixel in ; represents the image I generated by the image illumination simulation model gen The value corresponding to the i-th pixel in L perc represents the perceived loss; N′ p Indicates I gt The number of pixels in the feature map extracted by the pre-trained convolutional neural network; Indicates I gt The value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; Indicates I gen The value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; L adv Denotes the adversarial loss; D(I gen ) indicates I gen The value obtained after being processed by the discriminator; E[·] represents the expected value of all training samples.

[0092] The lighting simulation model is optimized by reconstruction loss, perceptual loss and adversarial loss. The design of the comprehensive loss function enables the lighting simulation image to maintain details and have realism. The adversarial loss improves the resolution and quality consistency of the images generated by the model.

[0093] Furthermore, the discriminator is composed of a convolutional layer and a fully connected layer; after updating the image illumination simulation model parameters m times, the discriminator parameters are updated once; the image illumination simulation model parameters are updated by calculating the image illumination simulation model loss, and the image illumination simulation model loss calculation formula is:

[0094]

[0095] in, represents the image illumination simulation model loss; Represents the average reconstruction loss of all training samples; represents the average perceptual loss of all training samples; λ1 and λ2 represent and L adv The weight coefficient of

[0096] The discriminator parameters are updated by calculating the discriminator loss, and the discriminator loss calculation formula is:

[0097] L D = -E[log(D(I gt ))+log(1-D(I gen ))];

[0098] Among them, D(I gt ) represents the real image I gt The value obtained after being processed by the discriminator.

[0099] The introduction of the discriminator makes the images generated by the illumination simulation model closer to the real images and can effectively distinguish the feature differences between the real images and the generated images. The alternating training strategy is adopted to avoid the performance imbalance between the image illumination simulation model and the discriminator during the training process. The weighted combination of reconstruction loss, perceptual loss and adversarial loss realizes multi-faceted optimization of the generated image quality.

[0100] Further, the state detection model includes a second input processing layer, a feature extraction layer, a feature fusion layer and a dual-branch output layer;

[0101] The second input processing layer normalizes the first illumination simulation image to be detected, the second illumination simulation image to be detected, the third illumination simulation image to be detected, and the napped woven fabric image to be detected and splices them in terms of the number of channels to obtain an initial comprehensive feature map;

[0102] The feature extraction layer adopts a ResNet-50 structure, introduces a channel attention mechanism in each residual block of ResNet-50, and extracts first comprehensive feature maps of different scales according to the initial comprehensive feature map;

[0103] The feature fusion layer adopts a multi-scale feature pyramid structure, and fuses features of different levels according to the first comprehensive feature map to obtain a second comprehensive feature map of different scales;

[0104] The dual-branch output layer obtains a state detection result according to the second comprehensive feature map of different scales; the dual-branch output layer includes a defect detection branch and a state scoring branch; the state detection result includes a defect detection result vector and a state score; the defect detection branch outputs the defect detection result vector; the state scoring branch outputs the state score.

[0105] In an embodiment of the present application, the feature extraction layer based on the ResNet-50 structure has four stages, and the first comprehensive feature maps of four scales are obtained respectively; the first comprehensive feature maps of three scales in the second, third and fourth stages are input into the feature fusion layer based on the multi-scale feature pyramid structure, and the second comprehensive feature maps of three scales are obtained by fusing features of different levels through upsampling paths and downsampling paths.

[0106] The second input processing layer normalizes and splices images under multiple lighting conditions, providing rich input information; the ResNet-50 structure that introduces the channel attention mechanism can self-focus on important feature channels and improve the effectiveness of feature extraction; the multi-scale feature pyramid structure realizes the fusion of features at different levels, which helps to detect fabric defects of different scales; the dual-branch output enables the model to complete the two tasks of defect recognition and state assessment at the same time, improving the practicality of the model.

[0107] Furthermore, the channel attention mechanism includes:

[0108] Generate a global feature vector for each channel by global average pooling of the input feature map;

[0109] Generate channel attention weights through a multilayer perceptron according to the global feature vector;

[0110] Multiplying the channel attention weight by the input feature map of the channel attention mechanism element by element in the channel dimension to generate a channel weighted feature map;

[0111] The channel weighted feature map is added to the input feature map of the residual block through a skip connection mechanism to obtain an output feature map of each residual block.

[0112] The channel attention mechanism improves the model's ability to focus on key feature channels and improves the effectiveness of feature extraction; the skip connection mechanism avoids the problem of information loss in the process of deep feature extraction and enhances the model's ability to capture complex fabric states.

[0113] Furthermore, the state detection model loss includes defect detection loss and state scoring loss, and the calculation formula is as follows:

[0114]

[0115] Among them, L det represents the defect detection loss; L focal represents the defect classification loss calculated by focal loss; L giou represents the defect localization loss calculated by the intersection-over-union loss; and represents weight; represents the state detection model loss; L score Denotes the state score loss.

[0116] Furthermore, the defect classification loss, the defect location loss and the state scoring loss are calculated as follows:

[0117]

[0118] Where K represents the number of defect categories; α k represents the balance factor of the kth category; N b represents the total number of prediction boxes; p j,k represents the predicted probability that the j-th prediction box belongs to the k-th category; γ represents the focus parameter; IoU j A represents the intersection-over-union ratio of the j-th predicted box and the corresponding true box; j represents the minimum enclosed area that contains both the j-th predicted box and the corresponding true box; B j represents the union area of ​​the jth predicted box and the corresponding true box; s gen Represents the predicted status score; s gt represents the true state score; δ represents the error threshold.

[0119] The focal loss is used to calculate the defect classification loss, which can effectively solve the sample imbalance problem and improve the detection capability of rare defect types. The defect location loss is calculated by the intersection-over-union loss, which improves the accuracy of defect location prediction. The design of the state scoring loss takes into account the continuity characteristics of the scoring, which helps to improve the accuracy and reliability of the scoring results and makes the scoring more consistent with the actual fabric state.

[0120] Table 2 Defect detection performance

[0121] Defect Category Accuracy (%) Recall rate (%) F1 score (%) Villus loss 90.2 88.5 89.3 Too dense villi 86.4 84.8 85.6 Abnormal villus orientation 83.5 81.2 82.3 Fabric damage 89.6 87.3 88.4 Pilling 85.8 83.6 84.7

[0122] In the embodiment of the present application, on the test set divided from the second data set, the detection performance of the state detection model for some defect categories is shown in Table 2, indicating that the method of the present invention has good detection capabilities for various types of napped woven fabric defects.

[0123] Table 3 Defect detection performance

[0124] Evaluation Metrics value Mean absolute error 0.385 Root mean square error 0.462 <![CDATA[R 2 Coefficient of determination]]> 0.926

[0125] The state scoring performance of the state detection model is shown in Table 3, which proves that the proposed method has high accuracy and stability in fabric state assessment.

[0126] The method for detecting the state of napped woven fabrics based on machine vision firstly constructs an image illumination simulation model, expands a training data set by designing a variety of illumination conditions, and enhances the adaptability of the model to different illumination environments; at the same time, a multi-level downsampling-upsampling structure and an illumination condition injection mechanism are adopted to ensure the authenticity of the simulated image; in the state detection model, features are extracted by introducing a ResNet-50 structure with a channel attention mechanism, and feature fusion is performed using a multi-scale feature pyramid structure, and finally defect detection and state scoring are simultaneously realized through dual-branch output, which can not only accurately locate and classify fabric defects, but also quantitatively evaluate the overall state of the fabric; the method solves the problem of a single illumination environment in the state detection of napped woven fabrics in an actual production process, and improves the robustness while ensuring the detection accuracy.

[0127] Embodiment 2

[0128] The present invention also provides a system for detecting the state of napped woven fabrics based on machine vision, and applies a method for detecting the state of napped woven fabrics based on machine vision, such as Figure 3 As shown, including:

[0129] A lighting simulation design module, which designs a first lighting condition, a second lighting condition and a third lighting condition through lighting parameters;

[0130] A data set acquisition module acquires N images of napped woven fabrics and annotates them with state labels to obtain a first data set;

[0131] A simulation image generation module is used to construct an image illumination simulation model, input the first illumination condition, the second illumination condition, the third illumination condition and the napped woven fabric image into the image illumination simulation model, generate a first illumination simulation image, a second illumination simulation image and a third illumination simulation image, and traverse all the napped woven fabric images in the first data set to obtain a second data set of 4N in number;

[0132] A state detection design module is used to construct a state detection model and train the state detection model using the second data set;

[0133] The real-time state detection module obtains the image of the napped woven fabric to be detected, inputs the first lighting condition, the second lighting condition, the third lighting condition and the image of the napped woven fabric to be detected into the image lighting simulation model, generates the first lighting simulation image to be detected, the second lighting simulation image to be detected and the third lighting simulation image to be detected; inputs the first lighting simulation image to be detected, the second lighting simulation image to be detected, the third lighting simulation image to be detected and the image of the napped woven fabric to be detected into the trained state detection model, and obtains the state detection result of the napped woven fabric.

[0134] Furthermore, the image illumination simulation model includes a first input processing layer, a downsampling layer, an upsampling layer and an output processing layer; the first input processing layer performs pixel value normalization processing on the input image to obtain a first feature map, and maps the illumination conditions to d embed dimensional space to obtain an illumination condition vector; the downsampling layer includes n downsampling blocks, through which the first feature map is downsampled, multi-level features are extracted, and a second feature map is obtained; the upsampling layer includes n upsampling blocks and n illumination condition injection blocks, through which the second feature map is upsampled, and the illumination condition vector is fused with the output of the upsampling block through the illumination condition injection block to obtain a third feature map; the output processing layer performs convolution and denormalization on the third feature map to generate an illumination simulation image.

[0135] Furthermore, the image illumination simulation model loss includes reconstruction loss, perception loss and adversarial loss, and the calculation formula is as follows:

[0136]

[0137] L adv = -E[log(D(I gen ))];

[0138] Among them, L rec represents the reconstruction loss; N p Indicates the number of pixels of the input image; Represents the real image I gt The value corresponding to the i-th pixel in ; represents the image I generated by the image illumination simulation model gen The value corresponding to the i-th pixel in L perc represents the perceived loss; N′ p Indicates I gt The number of pixels in the feature map extracted by the pre-trained convolutional neural network; Indicates I gtThe value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; Indicates I gen The value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; L adv Denotes the adversarial loss; D(I gen ) indicates I gen The value obtained after being processed by the discriminator; E[·] represents the expected value of all training samples.

[0139] Furthermore, the discriminator is composed of a convolutional layer and a fully connected layer; after updating the image illumination simulation model parameters m times, the discriminator parameters are updated once; the image illumination simulation model parameters are updated by calculating the image illumination simulation model loss, and the image illumination simulation model loss calculation formula is:

[0140]

[0141] in, represents the image illumination simulation model loss; Represents the average reconstruction loss of all training samples; represents the average perceptual loss of all training samples; λ1 and λ2 represent and L adv The weight coefficient of

[0142] The discriminator parameters are updated by calculating the discriminator loss, and the discriminator loss calculation formula is:

[0143] L D = -E[log(D(I gt ))+log(1-D(I gen ))];

[0144] Among them, D(I gt ) represents the real image I gt The value obtained after being processed by the discriminator.

[0145] In the embodiment of the present application, the image illumination simulation model data set is obtained by photographing fabric samples in a standard illumination box using 20 different illumination parameters; the 20 illumination parameters include illumination parameters of the first illumination condition, the second illumination condition, and the third illumination condition; during the training process, the illumination parameters are used as illumination conditions and the corresponding real images are input into the image illumination simulation model; the model is enabled to learn the mapping relationship between illumination parameters and image performance, thereby simulating any illumination condition. The evaluation of the image illumination simulation effect on the test set is shown in Table 4, which shows that the simulated image generated by the image illumination simulation model has a high consistency with the real image at the pixel level, and maintains the structural characteristics of the fabric.

[0146] Table 4 Evaluation of image illumination simulation effect

[0147] Evaluation Metrics First lighting condition Second lighting condition The third lighting condition Peak signal-to-noise ratio (db) 32.45 31.98 31.56 Structural Similarity Index 0.956 0.944 0.938

[0148] Further, the state detection model includes a second input processing layer, a feature extraction layer, a feature fusion layer and a dual-branch output layer;

[0149] The second input processing layer normalizes the first illumination simulation image to be detected, the second illumination simulation image to be detected, the third illumination simulation image to be detected, and the napped woven fabric image to be detected and splices them in terms of the number of channels to obtain an initial comprehensive feature map;

[0150] The feature extraction layer adopts a ResNet-50 structure, introduces a channel attention mechanism in each residual block of ResNet-50, and extracts first comprehensive feature maps of different scales according to the initial comprehensive feature map;

[0151] The feature fusion layer adopts a multi-scale feature pyramid structure, and fuses features of different levels according to the first comprehensive feature map to obtain a second comprehensive feature map of different scales;

[0152] The dual-branch output layer obtains a state detection result according to the second comprehensive feature map of different scales; the dual-branch output layer includes a defect detection branch and a state scoring branch; the state detection result includes a defect detection result vector and a state score; the defect detection branch outputs the defect detection result vector; the state scoring branch outputs the state score.

[0153] Furthermore, the state detection model loss includes defect detection loss and state scoring loss, and the calculation formula is as follows:

[0154]

[0155] Among them, L det represents the defect detection loss; L focal represents the defect classification loss calculated by focal loss; L giou represents the defect localization loss calculated by the intersection-over-union loss; and represents the weight coefficient; represents the state detection model loss; L score Denotes the state score loss.

[0156] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting the state of napped woven fabric based on machine vision, characterized in that: include: Designing a first lighting condition, a second lighting condition, and a third lighting condition through lighting parameters; The lighting parameters include light source type, color temperature, illumination and incident angle; Obtain N images of napped woven fabrics and annotate them with status labels to obtain a first data set; the status labels include fabric defect labels and fabric status score labels; the fabric defect labels include defect locations and defect categories; Constructing an image lighting simulation model, inputting the first lighting condition, the second lighting condition, the third lighting condition and the napped woven fabric image into the image lighting simulation model, generating a first lighting simulation image, a second lighting simulation image and a third lighting simulation image, and traversing all the napped woven fabric images in the first data set to obtain a second data set of 4N in number; Building a state detection model, and training the state detection model using the second data set; Acquire the image of the napped woven fabric to be detected, input the first lighting condition, the second lighting condition, the third lighting condition and the image of the napped woven fabric to be detected into the image lighting simulation model, and generate a first lighting simulation image to be detected, a second lighting simulation image to be detected and a third lighting simulation image to be detected; The first illumination simulation image to be detected, the second illumination simulation image to be detected, the third illumination simulation image to be detected and the napped woven fabric image to be detected are input into the trained state detection model to obtain the napped woven fabric state detection result.

2. The method for detecting the state of napped woven fabric based on machine vision according to claim 1, characterized in that: The image illumination simulation model includes a first input processing layer, a downsampling layer, an upsampling layer and an output processing layer; the first input processing layer performs pixel value normalization processing on the input image to obtain a first feature map, and maps the illumination conditions to d embed dimensional space to obtain an illumination condition vector; the downsampling layer includes n downsampling blocks, through which the first feature map is downsampled, multi-level features are extracted, and a second feature map is obtained; the upsampling layer includes n upsampling blocks and n illumination condition injection blocks, through which the second feature map is upsampled, and the illumination condition vector is fused with the output of the upsampling block through the illumination condition injection block to obtain a third feature map; the output processing layer performs convolution and denormalization on the third feature map to generate an illumination simulation image.

3. The method for detecting the state of napped woven fabric based on machine vision according to claim 2, characterized in that: The image illumination simulation model further includes a skip connection mechanism, which transmits the output of the down-sampling block to the up-sampling block of the corresponding level.

4. The method for detecting the state of napped woven fabric based on machine vision according to claim 2, characterized in that: The lighting condition injection block includes: The output of the i-th upsampling block is reshaped to obtain a feature map vector, which is expressed as: Among them, F i Represents the feature map output by the i-th upsampling block; Indicates F i The reshaped feature map vector; C i Indicates F i The number of channels; H i and W i Indicates F i The height and width of ; R represents a real number; The conditional attention weight is calculated according to the feature map vector and the illumination condition vector, and the calculation formula is: Among them, a i represents the conditional attention weight of the i-th illumination condition injection block; softmax represents a normalization function; express The transpose of W e A projection matrix representing the illumination condition vector e; The conditional attention weight is multiplied by the feature map vector and the feature map is reshaped to obtain the third feature map.

5. The method for detecting the state of napped woven fabric based on machine vision according to claim 1, characterized in that: The image illumination simulation model loss includes reconstruction loss, perception loss and adversarial loss, and the calculation formula is as follows: L adv =-E[log(D(I gen ))]; Among them, L rec represents the reconstruction loss; N p Indicates the number of pixels of the input image; Represents the real image I gt The value corresponding to the i-th pixel in ; represents the image I generated by the image illumination simulation model gen The value corresponding to the i-th pixel in L perc represents the perceived loss; N′ p Indicates I gt The number of pixels in the feature map extracted by the pre-trained convolutional neural network; Indicates I gt The value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; Indicates I gen The value corresponding to the jth pixel in the feature map extracted by the pre-trained convolutional neural network; L adv Denotes the adversarial loss; D(I gen ) indicates I gen The value obtained after being processed by the discriminator; E[·] represents the expected value of all training samples.

6. The method for detecting the state of napped woven fabric based on machine vision according to claim 5, characterized in that: The discriminator is composed of a convolutional layer and a fully connected layer; after updating the image illumination simulation model parameters m times, the discriminator parameters are updated once; the image illumination simulation model parameters are updated by calculating the image illumination simulation model loss, and the image illumination simulation model loss calculation formula is: in, represents the image illumination simulation model loss; Represents the average reconstruction loss of all training samples; represents the average perceptual loss of all training samples; λ1 and λ2 represent and L adv The weight coefficient of The discriminator parameters are updated by calculating the discriminator loss, and the discriminator loss calculation formula is: L D =-E[log(D(I gt ))+log(1-D(I gen ))]; Among them, D(I gt ) represents the real image I gt The value obtained after being processed by the discriminator.

7. The method for detecting the state of napped woven fabric based on machine vision according to claim 1, characterized in that: The state detection model includes a second input processing layer, a feature extraction layer, a feature fusion layer and a dual-branch output layer; The second input processing layer normalizes the first illumination simulation image to be detected, the second illumination simulation image to be detected, the third illumination simulation image to be detected, and the napped woven fabric image to be detected and splices them in terms of the number of channels to obtain an initial comprehensive feature map; The feature extraction layer adopts a ResNet-50 structure, introduces a channel attention mechanism in each residual block of ResNet-50, and extracts first comprehensive feature maps of different scales according to the initial comprehensive feature map; The feature fusion layer adopts a multi-scale feature pyramid structure, and fuses features of different levels according to the first comprehensive feature map to obtain a second comprehensive feature map of different scales; The dual-branch output layer obtains a state detection result according to the second comprehensive feature map of different scales; the dual-branch output layer includes a defect detection branch and a state scoring branch; The state detection result includes a defect detection result vector and a state score; The defect detection branch outputs the defect detection result vector; the state scoring branch outputs the state score.

8. The method for detecting the state of napped woven fabric based on machine vision according to claim 7, characterized in that: The channel attention mechanism includes: Generate a global feature vector for each channel by global average pooling of the input feature map; Generate channel attention weights through a multilayer perceptron according to the global feature vector; Multiplying the channel attention weight by the input feature map of the channel attention mechanism element by element in the channel dimension to generate a channel weighted feature map; The channel weighted feature map is added to the input feature map of the residual block through a skip connection mechanism to obtain an output feature map of each residual block.

9. The method for detecting the state of napped woven fabric based on machine vision according to claim 1, characterized in that: The state detection model loss includes defect detection loss and state scoring loss, and the calculation formula is as follows: Among them, L det represents the defect detection loss; L focal represents the defect classification loss calculated by focal loss; L giou represents the defect localization loss calculated by the intersection-over-union loss; and represents the weight coefficient; represents the state detection model loss; L score Denotes the state score loss.

10. A machine vision-based napped woven fabric state detection system, characterized in that: include: A lighting simulation design module, which designs a first lighting condition, a second lighting condition and a third lighting condition through lighting parameters; A data set acquisition module acquires N images of napped woven fabrics and annotates them with state labels to obtain a first data set; A simulation image generation module is used to construct an image illumination simulation model, input the first illumination condition, the second illumination condition, the third illumination condition and the napped woven fabric image into the image illumination simulation model, generate a first illumination simulation image, a second illumination simulation image and a third illumination simulation image, and traverse all the napped woven fabric images in the first data set to obtain a second data set of 4N in number; A state detection design module is used to construct a state detection model and train the state detection model using the second data set; The real-time state detection module obtains the image of the napped woven fabric to be detected, inputs the first lighting condition, the second lighting condition, the third lighting condition and the image of the napped woven fabric to be detected into the image lighting simulation model, generates the first lighting simulation image to be detected, the second lighting simulation image to be detected and the third lighting simulation image to be detected; inputs the first lighting simulation image to be detected, the second lighting simulation image to be detected, the third lighting simulation image to be detected and the image of the napped woven fabric to be detected into the trained state detection model, and obtains the state detection result of the napped woven fabric.

Images