A knitted fabric defect detection method and system based on image processing

Through knitted image enhancement processing and feature fusion processing, the problem of insufficient accuracy of U-Net network when detecting small-scale defects is solved, and the accuracy and robustness of knitted defect detection are improved.

CN118781128BActive Publication Date: 2024-11-26SHAOXING BUJI KNITTING CO LTD
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
CN202411282313.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-11-26
Estimated Expiration
2044-09-13

AI Technical Summary

Technical Problem

U-Net networks are difficult to accurately capture defect characteristics at small scales in knitted defect detection, resulting in problems of missed inspection and missed inspection.

Method used

By enhancing the knitted images, multiple image features are extracted and the best feature combination is screened, and deep features extracted by neural networks are combined for fusion processing, which enhances the detection ability of micro-scale defects.

Benefits of technology

It improves the accuracy and robustness of defect detection of knitted fabrics, enhances the ability to extract micro-scale defect features, and reduces processing difficulty and computing resource consumption.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118781128B_ABST
    Figure CN118781128B_ABST
Patent Text Reader

Abstract

The present invention provides a knitwear defect detection method and system based on image processing, which relates to the technical field of image processing. The method comprises: obtaining a knitwear image; performing enhancement processing on the knitwear image; extracting a plurality of image features of the knitwear image after enhancement processing; screening the image features, and screening out the best feature combination from the plurality of image features; extracting the depth features of the knitwear image after enhancement processing through a neural network; fusing the image features in the best feature combination and the depth features extracted through the neural network to obtain a fusion feature map; and detecting defective areas in the knitwear image according to the fusion feature map. The present invention can improve the extensiveness and accuracy of feature extraction, enhance the ability to extract features of micro-scale defects, and improve the detection accuracy of micro-scale defects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a knitwear defect detection method and system based on image processing. Background Art

[0002] Due to factors such as yarn quality, equipment aging, improper process and environmental changes, various defects will be formed in the process of knitwear processing. The detection of knitwear defects is not only a key step to ensure product quality, but also can improve production efficiency, reduce costs, enhance market competitiveness, and support enterprises to move towards intelligent manufacturing. Therefore, the implementation of an effective defect detection mechanism has far-reaching strategic significance for knitwear manufacturers.

[0003] With the rapid development of technologies such as pattern recognition, machine vision, and deep learning, as well as the urgent need for knitwear defect detection, more and more modern technologies are being applied to knitwear defect detection. For example, the currently popular U-Net network. With its efficient convolutional architecture and innovative jump connections, the U-Net network has become one of the classic models in the field of image segmentation. Its ability to retain details and accurately segment makes it have great application potential in knitwear defect detection, and can meet the industry's needs for high precision, real-time performance, and efficient data utilization.

[0004] However, the U-Net network also has some limitations in the actual detection of knitwear defects. Although the U-Net retains low-level detail information through jump connections, very small targets or subtle structures are still easily lost during the downsampling process and cannot be accurately captured, resulting in weak feature extraction capabilities for micro-scale defects, making it difficult to accurately detect micro-scale defects, and prone to false detection and missed detection. Summary of the invention

[0005] In order to solve the technical problem that very small targets or subtle structures are still easily lost in the downsampling process of current technology and cannot be accurately captured, resulting in weak ability to extract features of tiny-scale defects, difficulty in accurately detecting tiny-scale defects, and prone to false detection and missed detection, the present invention provides a knitted fabric defect detection method and system based on image processing.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect:

[0008] An embodiment of the present invention provides a knitwear defect detection method based on image processing, comprising:

[0009] S1: Acquire knitwear images;

[0010] S2: performing enhancement processing on the knitwear image;

[0011] S3: extracting various image features of the enhanced knitwear image;

[0012] S4: screening the image features to select the best feature combination from the multiple image features;

[0013] S5: Extracting deep features of the enhanced knitwear image through a neural network;

[0014] S6: fusing the image features in the best feature combination and the deep features extracted by the neural network to obtain a fused feature map;

[0015] S7: Detect defective areas in the knitwear image according to the fused feature map.

[0016] Second aspect:

[0017] An embodiment of the present invention provides a knitwear defect detection system based on image processing, comprising:

[0018] processor;

[0019] A memory having computer-readable instructions stored therein, wherein when the computer-readable instructions are executed by the processor, the knitwear defect detection method based on image processing as described in the first aspect is implemented.

[0020] The third aspect:

[0021] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the knitwear defect detection method based on image processing as described in the first aspect is implemented.

[0022] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0023] In the present invention, the knitted fabric image is first enhanced to enhance the recognition of micro-scale defects, and then the macro-scale image features of the knitted fabric image are fused with the deep features extracted by the neural network to improve the breadth and accuracy of feature extraction, enhance the ability to extract micro-scale defect features, and improve the detection accuracy of micro-scale defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A schematic flow chart of a knitwear defect detection method based on image processing provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the structure of a knitwear defect detection method based on image processing provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of histogram clipping provided by an embodiment of the present invention;

[0028] Figure 4 A schematic structural diagram of a knitwear defect detection system based on image processing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0030] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0031] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0032] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0033] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0034] Reference Manual Attached Figure 1 , shows a schematic flow chart of a knitwear defect detection method based on image processing provided by an embodiment of the present invention.

[0035] Reference Manual Attached Figure 2 , shows a structural schematic diagram of a knitwear defect detection method based on image processing provided by an embodiment of the present invention.

[0036] The embodiment of the present invention provides a knitted fabric defect detection method based on image processing, which can be implemented by a knitted fabric defect detection device based on image processing, and the knitted fabric defect detection device based on image processing can be a terminal or a server. The processing flow of the knitted fabric defect detection method based on image processing may include the following steps:

[0037] S1: Acquire knitwear images.

[0038] Optionally, the knitted fabric image may be captured by a camera.

[0039] S2: Enhance the knitwear image.

[0040] In a possible implementation, S2 specifically includes sub-steps S201 to S206:

[0041] S201: Obtaining an original histogram of a knitwear image.

[0042] S202: Using the average grayscale value in the original histogram as a segmentation point, the original histogram is segmented into a low sub-histogram and a high sub-histogram.

[0043] It should be noted that by segmenting the histogram of the original image and processing the low and high sub-histograms separately, the contrast can be improved in the dark and bright areas of the image. This processing method can better stretch the grayscale distribution, making the details that may have been difficult to distinguish become clearer.

[0044] Optionally, the average gray value is specifically:

[0045]

[0046] in, SP represents the average gray value in the original histogram, l k Indicates k Gray levels, n k Indicates k The number of pixels with gray levels, N Represents the total number of pixels in the image. L Indicates the total gray level.

[0047] S203: Three limit lines are set for the low sub-histogram and the high sub-histogram respectively:

[0048]

[0049] in, PL 1 means the first limit line of the low subhistogram, PL 2 represents the second limit line of the low subhistogram, PL 3 represents the third limit line of the low subhistogram, GR L1 represents the first gray level ratio of the low sub-histogram, GR L2 represents the second gray level ratio of the low sub-histogram, GR L3 represents the third gray level ratio of the low sub-histogram, D L represents the gray level ratio difference in the low sub-histogram, SP L represents the average gray value in the low sub-histogram, L min Represents the minimum grayscale value in the original histogram, PH 1 indicates the first limit line of the high subhistogram, PH 2 represents the second limit line of the high subhistogram, PH 3 represents the third limit line of the high subhistogram, GR H1 represents the first gray level ratio of the high subhistogram, GR H2 represents the second gray level ratio of the high sub-histogram, GR H3 represents the third gray level ratio of the high subhistogram, D H represents the gray level ratio difference in the high sub-histogram, SP H represents the average gray value in the high sub-histogram, L max Indicates the maximum grayscale value in the original histogram.

[0050] It should be noted that the main function of the limit line is to control the distribution of grayscale values ​​during image enhancement by performing specific cropping and redistribution of the histogram, thereby preventing the image from being over-saturated or having excessive contrast, thereby achieving a more balanced enhancement effect.

[0051] Reference Manual Attached Figure 3 , shows a schematic diagram of histogram clipping provided by an embodiment of the present invention.

[0052] S204: According to the limit lines, the low sub-histogram and the high sub-histogram are trimmed, the part below or equal to the first limit line is trimmed to the first limit line, the part between the first limit line and the third limit line is trimmed to the second limit line, and the part greater than or equal to the third limit line is trimmed to the third limit line.

[0053] It should be noted that by setting limit lines and clipping the histogram, it is possible to prevent the grayscale from being overly concentrated in certain specific ranges and avoid oversaturation of the image during the enhancement process. This can prevent some areas of the image from losing information due to excessively high or low brightness, and ensure the quality of the image after enhancement.

[0054] S205: Calculate the cumulative density function of the low sub-histogram and the high sub-histogram after the clipping process.

[0055] Among them, the cumulative density function (CDF) is an important concept in probability theory and statistics, which is used to describe the probability distribution of random variable values. By calculating the cumulative probability of each gray level and redistributing the gray value according to these cumulative probabilities, CDF can effectively enhance the image contrast, improve the image quality, and make the details more prominent.

[0056] S206: performing enhancement processing on the knitwear image according to the cumulative density function of the lower sub-histogram and the upper sub-histogram after the clipping processing.

[0057] It should be noted that by introducing the segmented cumulative density function and performing nonlinear adjustments on pixels in different grayscale ranges, the detailed information in the image can be better preserved and enhanced, especially the tiny defects that may exist on the surface of knitwear. This is crucial for detecting tiny defects in knitwear, because these defects are often hidden in areas with more concentrated grayscale distribution.

[0058] Optionally, S206 specifically includes: performing enhancement processing on the knitwear image according to the following formula:

[0059]

[0060] in, T ( k ) indicates that after enhancement k The gray value of the gray level, CDF L represents the cumulative density function of the low subhistogram, CDF L ( k ) represents the first k The cumulative density of gray levels, CDF H represents the cumulative density function of the high subhistogram, CDF H ( k ) represents the first k The cumulative density of gray levels, l min Represents the minimum gray value, l max Indicates the maximum grayscale value.

[0061] In the present invention, after the enhancement processing, the contrast and details of the knitwear image are improved, which enables subsequent image processing steps (such as segmentation, classification, and recognition) to be performed on a better image basis, thereby reducing the processing difficulty and improving the robustness and reliability of the algorithm.

[0062] S3: Extracting various image features of the enhanced knitwear image.

[0063] Optionally, the image features include: connection features, statistical features, distribution features, shape features, HOG features and LGP features.

[0064] Optionally, the connection feature is used to describe the connection relationship between pixels or the connectivity between regions in the image.

[0065] Optionally, the statistical feature is a feature extracted based on statistical information of pixel values ​​of the image. The statistical feature includes variance, mean value, and kurtosis.

[0066] Optionally, the distribution feature describes a distribution pattern of pixel values ​​in the image. The distribution feature includes a grayscale histogram, and the grayscale distribution of the image is determined by the grayscale histogram.

[0067] Optionally, shape features are used to describe the geometric shape and boundary information of objects in the image. Shape features can be performed on a grid basis, where the shape feature value is set to 1 if a cell of the grid is partially or completely covered by the shape, and is set to 0 if the cell is outside the boundary.

[0068] Optionally, the HOG feature describes the distribution of local gradient directions in the image. The HOG feature describes the texture and shape characteristics of the image by counting the distribution of gradient directions in the local area of ​​the image. How to extract the HOG feature is already a very existing technology, and the present invention will not repeat it in detail.

[0069] Optionally, the LGP feature is a kind of local texture feature, which is based on the local gradient direction pattern of image pixels and is used to describe the detail information of the image. How to extract the LGP feature is already a very existing technology, and the present invention will not repeat it in detail.

[0070] S4: Screen the image features and select the best feature combination from multiple image features.

[0071] In a possible implementation, S4 specifically includes: screening the image features by principal component analysis or particle swarm optimization algorithm, and screening out the best feature combination from multiple image features.

[0072] Alternatively, principal component analysis (PCA) is a commonly used dimensionality reduction technique that transforms the original feature space into a new space through linear transformation, maximizing the variance of the data in the new space, thereby extracting the most important features (i.e., principal components). Through PCA, the most representative set of features of the image can be screened out, thereby reducing the dimension of the data and improving the efficiency of the algorithm.

[0073] Specifically, we can first standardize the data, then calculate the covariance matrix between each image feature, and solve the eigenvalue and eigenvector of the covariance matrix. The larger the eigenvalue, the larger the variance of the data in the direction indicated by the corresponding eigenvector, and the more important the feature corresponding to this direction. k The eigenvectors corresponding to the largest eigenvalues ​​are taken as the main features and form the best feature combination.

[0074] Alternatively, Particle Swarm Optimization (PSO) is an optimization algorithm based on swarm intelligence, which simulates the social behavior of biological groups such as bird flocks and fish schools, and finds the optimal solution to the problem through information sharing between individuals (particles) in the group.

[0075] Furthermore, the defect detection loss function can be constructed:

[0076]

[0077] in, L represents the loss function, L C represents the cross entropy loss, l 1 represents the weight coefficient of cross entropy loss, L conf represents the confidence loss, l 2 represents the weight coefficient of confidence loss, L IOU represents the detection box loss, l 3 represents the weight coefficient of the detection box loss, i represents the model parameters, represents the calculation of the second norm, represents the regularization term, l 4 represents the weight coefficient of the regularization term.

[0078] It should be noted that the cross entropy loss is used to optimize the classification accuracy and ensure that the model can correctly distinguish between defective and non-defective areas. The confidence loss is used to optimize the confidence score of the detection results, so that the model can effectively evaluate the confidence of the detection results when outputting the prediction. The detection box loss is used to optimize the position and size of the prediction box and increase the overlap between the prediction box and the true box, thereby enhancing the positioning accuracy of the model. The regularization term is used to prevent the model from overfitting and ensure that the model performs well on both training data and test data.

[0079] In the present invention, a multi-objective loss function is constructed by integrating cross entropy loss, confidence loss, detection box loss and regularization term, which can comprehensively optimize the performance of the fabric defect detection model. This method not only improves the accuracy of classification and positioning, but also enhances the robustness and generalization ability of the model, ensuring the stable performance of the model in different application scenarios.

[0080] Among them, those skilled in the art can set the weight coefficient of the cross entropy loss according to actual conditions. l 1. Weight coefficient of confidence loss l 2. Weight coefficient of detection box loss l 3 and the weight coefficient of the regularization term l The size of 4 is not limited in the present invention.

[0081] Specifically, the specific steps of screening image features through the particle swarm optimization algorithm and selecting the best feature combination from multiple image features include:

[0082] The inverse of the defect detection loss function is used as the fitness function of the particle swarm optimization algorithm.

[0083] Initialize the particle swarm, which contains multiple particles, each of which represents a feature combination.

[0084] Calculate the fitness value of each particle.

[0085] The position of the particle with the highest fitness value is determined as the global optimal position.

[0086] The position of each particle is updated according to the following formula:

[0087]

[0088] in, Indicates i The particle in t The particle position at iteration +1, Indicates i The particle in t The displacement vector at the iteration, exp represents the exponential function with the natural constant as the base,s Represents the shape parameter, which is a random number between -1 and 1. r represents the logarithmic spiral constant, P best represents the global optimal position, l represents the attraction factor, Indicates t The global optimal position at the iteration.

[0089] In the present invention, the spiral mechanism is introduced to make the search path of particles more complex and diversified. Such a complex search path helps the particle swarm to conduct a more comprehensive exploration in the search space and avoid falling into a local optimum.

[0090] In the present invention, an attraction factor is introduced to ensure that particles are always guided by the global optimal position, thus guaranteeing the convergence of the search.

[0091] The distance vector is specifically:

[0092]

[0093] Where MMI represents mutual information, sim Represents the Euclidean distance similarity.

[0094] In the present invention, by judging whether the similarity (sim) between the particle and the global optimal position is less than 0.5, the search direction and strength of the particle can be dynamically adjusted. When the similarity is less than 0.5, the particle will move directly to the global optimal position to ensure rapid convergence. When the similarity is greater than or equal to 0.5, the particle will adjust its position according to the mutual information, which can capture the complex relationship between features, promote fine-grained search and optimization of particles in high-similarity areas, enhance local development capabilities, and increase search diversity.

[0095] The mutual information is specifically:

[0096]

[0097] in, H represents information entropy, Indicates i The particle in t The information entropy of the particle position at the iteration, Indicates t The information entropy of the global optimal position at the iteration is Indicates i The particle in t The joint information entropy with the global optimal position at the iteration.

[0098] It should be noted that the introduction of mutual information can effectively measure the correlation between features. Using mutual information to adjust the position of particles helps to reduce redundancy in feature selection and improve the independence and information content of the selected features.

[0099] Determine whether the maximum number of iterations has been reached. If so, output the feature combination represented by the particle with the highest fitness value as the best feature combination; otherwise, return to continue iterating.

[0100] In the present invention, the image features are screened by the particle swarm optimization algorithm, which can significantly improve the performance and efficiency of the model, reduce the consumption of computing resources, and automate the feature selection process.

[0101] S5: Extract deep features of enhanced knitwear images through neural networks.

[0102] In a possible implementation, S5 specifically includes sub-steps S501 to S504:

[0103] S501: Extract the initial features of the enhanced knitwear image through the convolution layer:

[0104]

[0105] in, X 1 represents the initial feature, RELU represents the RELU activation function, Conv1 represents the first convolution unit, X Represents a knitted fabric image.

[0106] It should be noted that the convolution layer can effectively capture local features in the image, such as edges, textures, etc., through convolution operations. Through multiple layers of convolution, the network extracts higher-level and more abstract features layer by layer, which helps to identify and analyze complex patterns and defects in knitwear images.

[0107] Optionally, you can set multiple types of convolutional layers for feature extraction. By setting convolution kernels of different sizes (such as 3x3, 5x5, 7x7, etc.), the model can capture information of different scales. Small convolution kernels can finely capture local features of the image, such as edges and textures; large convolution kernels can capture a wider range of contextual information, helping the model understand larger structures or patterns.

[0108] S502: Through the pooling layer, the extracted initial features are reduced in dimension to obtain pooled features:

[0109]

[0110] in, X 2 represents pooling features, and maxpolling represents maximum pooling processing.

[0111] It should be noted that by performing dimensionality reduction processing on the initial features through pooling operations (such as maximum pooling), the size of the feature map can be effectively reduced, thereby reducing the computational complexity. The pooling layer can also condense the features, retain the most important feature information, reduce noise and unimportant details, and thus improve the feature extraction and recognition capabilities of subsequent layers.

[0112] S503: Add attention weights to each channel through the attention layer.

[0113] Optionally, S503 specifically includes:

[0114] S5031: compress the feature map:

[0115]

[0116] in, y c represents the c-dimensional feature after compression processing, S represents the compression function, X represents the input feature map, H represents the height of the feature map, W Represents the width of the feature map, and xc represents the c-dimensional feature of the input feature map.

[0117] It should be noted that compression processing before assigning attention weights can effectively extract global information and ensure that the attention mechanism can better capture the overall importance of each feature channel.

[0118] S5032: Add attention weights to each channel by compressing the fully connected layer and reconstructing the fully connected layer:

[0119]

[0120] in, E represents the attention weight function, Y represents the feature map after compression processing, W s represents the reconstruction of the fully connected layer, W r Represents a compressed fully connected layer, Sigmoid represents the Sigmoid activation function, and ReLU represents the ReLU activation function.

[0121] S5033: Multiply the feature map according to the attention weight function to highlight the target area and suppress irrelevant areas.

[0122] It should be noted that by adjusting the attention weight of the feature map, the important feature areas in the image, such as possible defects on the surface of knitwear, can be effectively highlighted. The features of these key areas play a decisive role in subsequent classification or detection tasks. At the same time, those unimportant or irrelevant features can be suppressed, which helps to reduce the impact of noise on the model and avoid the model making wrong judgments due to irrelevant information.

[0123] In the present invention, the attention mechanism can automatically focus the model on more important feature areas in the image by adding weights to each channel. This is particularly critical for detecting small defects in knitwear images, because the attention mechanism can increase the model's sensitivity to these details, thereby improving the accuracy and robustness of defect detection.

[0124] S504: Through the fully connected layer, the feature maps output by each convolution layer and pooling layer are summarized to obtain deep features.

[0125] In the present invention, the deep features of the enhanced knitwear images are extracted, which can significantly improve the model's ability to understand and distinguish images. In particular, the introduction of the attention mechanism further enhances the model's ability to focus on key details, and the deep features finally output can provide strong support for subsequent analysis and decision-making.

[0126] S6: The image features in the best feature combination and the deep features extracted by the neural network are fused to obtain a fused feature map.

[0127] In a possible implementation, the fusion feature map is specifically:

[0128]

[0129] in, X mix represents the fused feature map, X i Represents the best feature combination i The feature map constructed by image features, l i represents the best feature combination i The weight coefficient of the image features, X h represents the feature map constructed by deep features, l h represents the weight coefficient of the deep feature, n Represents the total number of image features in the best feature combination.

[0130] Among them, those skilled in the art can set the size of the weight coefficient of each image feature and depth feature according to actual conditions, and the present invention does not limit this.

[0131] It should be noted that traditional image features (such as shape, texture, color, etc.) may have better expressive power for certain specific visual patterns, while deep features can usually capture higher-level and more abstract semantic information. By fusing these two types of features, the model can make full use of the advantages of different feature types and improve the overall feature expressive power.

[0132] In the present invention, the macro-scale image features of the knitwear image and the deep features extracted by the neural network are fused to improve the breadth and accuracy of feature extraction, enhance the ability to extract features of micro-scale defects, and improve the detection accuracy of micro-scale defects.

[0133] S7: Detect defective areas in the knitwear image according to the fused feature map.

[0134] In a possible implementation, S7 specifically includes sub-steps S701 and S702:

[0135] S701: Calculate the probability that each pixel in the knitwear image belongs to a defect category according to the fused feature map:

[0136]

[0137] in, P represents the probability of belonging to the defect category, Softmax represents the Softmax activation function, W represents the classification weight matrix, X mix represents the fused feature map, b Represents the bias term.

[0138] S702: Determine a defective area in the knitted fabric image according to the probability that each pixel point belongs to a defective category.

[0139] Specifically, the thresholding method can be used to compare the defect probability of each pixel with the threshold to generate a binary mask. The processed binary mask is subjected to connected region analysis to identify and mark each connected defect region. For each connected region, its minimum bounding box is calculated as the bounding box to obtain the location information and size of the defect region.

[0140] Furthermore, the model is trained by gradient descent method according to the defect detection loss function.

[0141] Specifically, the model parameters are optimized by gradient descent method:

[0142]

[0143] in, i t+1 Indicates t +Model parameters at 1 iteration, i t Indicates t The model parameters at the iteration, or t Indicates t The adaptive learning rate at the iteration, Represents the loss function L Network parameters i t The gradient of β represents the momentum coefficient.

[0144] In the present invention, momentum optimization can utilize the gradient accumulation effect of the previous iterations to speed up the parameter update on the long gradient path. Especially when approaching the convergence point, momentum can help the model jump out of the local optimal point faster and move towards the global optimal direction. The introduction of momentum terms can help the model reduce the oscillation phenomenon in the parameter update process when the gradient direction is unstable or the noise is large, thereby making the training process smoother and more stable.

[0145] Optionally, the adaptive learning rate is calculated as:

[0146]

[0147] in, or min represents the minimum learning rate, or max represents the maximum learning rate, T max represents the maximum number of iterations, and cos represents the cosine function.

[0148] In the present invention, as the training process proceeds, the learning rate is smoothly reduced from an initial larger value to a smaller value. This smooth decay method allows the model to quickly learn global features in the early stages of training, and refines the learning process in the later stages of training to avoid missing tiny optimal solutions. At the same time, due to the properties of the cosine function, the adjustment of the learning rate is very smooth, avoiding the training instability that may be caused by sudden changes in the learning rate. Compared with piecewise linear decay or exponential decay, cosine annealing can provide a more stable training process.

[0149] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0150] In the present invention, the knitted fabric image is first enhanced to enhance the recognition of micro-scale defects, and then the macro-scale image features of the knitted fabric image are fused with the deep features extracted by the neural network to improve the breadth and accuracy of feature extraction, enhance the ability to extract micro-scale defect features, and improve the detection accuracy of micro-scale defects.

[0151] Reference Manual Attached Figure 4 , showing a structural schematic diagram of a knitwear defect detection system based on image processing provided by the present invention.

[0152] The present invention further provides a knitwear defect detection system 20 based on image processing, which is applied to the above-mentioned knitwear defect detection method based on image processing, comprising:

[0153] Processor 201;

[0154] The memory 202 stores computer-readable instructions, and when the computer-readable instructions are executed by the processor 201, the knitwear defect detection method based on image processing as described in the method embodiment is implemented.

[0155] The knitwear defect detection system 20 based on image processing provided by the present invention can execute the above-mentioned knitwear defect detection method based on image processing and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0156] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0157] In the present invention, the knitted fabric image is first enhanced to enhance the recognition of micro-scale defects, and then the macro-scale image features of the knitted fabric image are fused with the deep features extracted by the neural network to improve the breadth and accuracy of feature extraction, enhance the ability to extract micro-scale defect features, and improve the detection accuracy of micro-scale defects.

[0158] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0159] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0160] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0161] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0162] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0163] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0164] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0169] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0170] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the knitwear defect detection method based on image processing as described in the method embodiment is implemented.

[0171] A computer-readable storage medium provided by the present invention can implement the steps and effects of the knitwear defect detection method based on image processing of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0172] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0173] In the present invention, the knitted fabric image is first enhanced to enhance the recognition of micro-scale defects, and then the macro-scale image features of the knitted fabric image are fused with the deep features extracted by the neural network to improve the breadth and accuracy of feature extraction, enhance the ability to extract micro-scale defect features, and improve the detection accuracy of micro-scale defects.

[0174] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0175] There are a few points to note:

[0176] (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0177] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the film or region is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a film, a region or a substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0178] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0179] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A knitwear defect detection method based on image processing, characterized in that: include: S1: Acquire knitwear images; S2: performing enhancement processing on the knitwear image; S3: extracting various image features of the enhanced knitwear image; S4: screening the image features to select the best feature combination from the multiple image features; S5: Extracting deep features of the enhanced knitwear image through a neural network; S6: fusing the image features in the best feature combination and the deep features extracted by the neural network to obtain a fused feature map; S7: Detecting defective areas in the knitwear image according to the fused feature map; Wherein, the S2 specifically includes: S201: Acquire an original histogram of the knitwear image; S202: Using the average grayscale value in the original histogram as a segmentation point, segmenting the original histogram into a low sub-histogram and a high sub-histogram; The average gray value is specifically: in, SP represents the average gray value in the original histogram, l k Indicates k Gray levels, n k Indicates k The number of pixels with gray levels, N Represents the total number of pixels in the image. L Indicates the total gray level; S203: setting three limit lines for the low sub-histogram and the high sub-histogram respectively: in, PL 1 means the first limit line of the low subhistogram, PL 2 represents the second limit line of the low subhistogram, PL 3 represents the third limit line of the low subhistogram, GR L1 represents the first gray level ratio of the low sub-histogram, GR L2 represents the second gray level ratio of the low sub-histogram, GR L3 represents the third gray level ratio of the low sub-histogram, D L represents the gray level ratio difference in the low sub-histogram, SP L represents the average gray value in the low sub-histogram, L min Represents the minimum grayscale value in the original histogram, PH 1 indicates the first limit line of the high subhistogram, PH 2 represents the second limit line of the high subhistogram, PH 3 represents the third limit line of the high subhistogram, GR H1 represents the first gray level ratio of the high subhistogram, GR H2 represents the second gray level ratio of the high sub-histogram, GR H3 represents the third gray level ratio of the high subhistogram, D H represents the gray level ratio difference in the high sub-histogram, SP H represents the average gray value in the high sub-histogram, L max Represents the maximum grayscale value in the original histogram; S204: According to the limit line, the low sub-histogram and the high sub-histogram are clipped, the portion lower than or equal to the first limit line is clipped to the first limit line, the portion between the first limit line and the third limit line is clipped to the second limit line, and the portion greater than or equal to the third limit line is clipped to the third limit line; S205: Calculating the cumulative density function of the low sub-histogram and the high sub-histogram after clipping; S206: performing enhancement processing on the knitwear image according to the cumulative density function of the lower sub-histogram and the upper sub-histogram after the clipping processing.

2. The knitwear defect detection method based on image processing according to claim 1, characterized in that: The S206 is specifically as follows: The knitwear image is enhanced according to the following formula: in, T ( k ) indicates that after enhancement k The gray value of the gray level, CDF L represents the cumulative density function of the low subhistogram, CDF L ( k ) represents the first k The cumulative density of gray levels, CDF H represents the cumulative density function of the high subhistogram, CDF H ( k ) represents the first k The cumulative density of gray levels, l min represents the minimum gray value, l max Indicates the maximum grayscale value.

3. The knitwear defect detection method based on image processing according to claim 1, characterized in that: The image features include: connection features, statistical features, distribution features, shape features, HOG features and LGP features.

4. The knitwear defect detection method based on image processing according to claim 3, characterized in that: The S4 is specifically: The image features are screened by principal component analysis or particle swarm optimization algorithm, and the best feature combination is screened out from the multiple image features.

5. The knitwear defect detection method based on image processing according to claim 1, characterized in that: The S5 specifically includes: S501: Extract the initial features of the enhanced knitwear image through the convolution layer: in, X 1 represents the initial feature, RELU represents the RELU activation function, Conv1 represents the first convolution unit, X Represents a knitwear image; S502: Through the pooling layer, the extracted initial features are subjected to dimensionality reduction processing to obtain pooling features: in, X 2 represents pooling features, and maxpolling represents maximum pooling processing; S503: adding attention weights to each channel through the attention layer; S504: Summarize the feature maps output by each convolutional layer and pooling layer through a fully connected layer to obtain the deep features.

6. The knitwear defect detection method based on image processing according to claim 5, characterized in that: The S503 specifically includes: S5031: compress the feature map: in, y c represents the c-dimensional feature after compression processing, S represents the compression function, X represents the input feature map, H represents the height of the feature map, W Represents the width of the feature map, and xc represents the c-dimensional feature of the input feature map; S5032: Add attention weights to each channel by compressing the fully connected layer and reconstructing the fully connected layer: in, E represents the attention weight function, Y represents the feature map after compression processing, W s represents the reconstruction of the fully connected layer, W r represents a compressed fully connected layer, Sigmoid represents the Sigmoid activation function, and ReLU represents the ReLU activation function; S5033: Multiply the feature map according to the attention weight function to highlight the target area and suppress irrelevant areas.

7. The knitwear defect detection method based on image processing according to claim 1, characterized in that: The fusion feature map is specifically: in, X mix represents the fused feature map, X i Represents the best feature combination i The feature map constructed by image features, λ i represents the best feature combination i The weight coefficient of the image features, X h represents the feature map constructed by deep features, λ h represents the weight coefficient of the deep feature, n Represents the total number of image features in the best feature combination.

8. The knitwear defect detection method based on image processing according to claim 1, characterized in that: The S7 specifically includes: S701: Calculate the probability that each pixel in the knitwear image belongs to a defect category according to the fused feature map: in, P represents the probability of belonging to the defect category, Softmax represents the Softmax activation function, W represents the classification weight matrix, X mix represents the fused feature map, b represents the bias term; S702: Determine a defective area in the knitwear image according to the probability that each pixel point belongs to a defective category.

9. A knitwear defect detection system based on image processing, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the knitwear defect detection method based on image processing as claimed in any one of claims 1 to 8 is implemented.

Citation Information

Patent Citations

  • A method of enhancing and identifying a traffic sign

    CN109800650A

  • Fingerprint and finger vein fusion identification method based on single near-infrared finger image

    CN110163123A

  • Cloth defect detection method based on convolutional neural network

    CN111210417A

  • Micro-seismic P-wave arrival time pickup method and device and storage medium

    CN112666604A

  • Image processing method and image processing system

    CN114549670A