Natural leather defect segmentation method based on MLP multi-layer neural network

Abstract

The invention discloses a natural leather defect segmentation method based on an MLP multi-layer neural network, comprising: obtaining a data set of leather images through an image acquisition module, dividing the data set into a training set and a test set; generating a leather color feature map according to different colors; using a feature vector as an input of the MLP multi-layer neural network, completing the construction of a leather defect segmentation model through feature training of the MLP multi-layer neural network; storing the model file; using a traditional image processing method to pre-process the leather image to be segmented, using the model to predict the leather image, and obtaining a defect segmentation map of the leather image; using a traditional image processing method to enrich the details of the defect segmentation map; extracting the outer contour and defect contour of the leather, and converting them into a DXF file. The invention is suitable for leather defect segmentation, can solve the technical problems of low segmentation efficiency and insufficient segmentation, reduce labor costs, and improve production efficiency.

Description

Technical Field

[0001] The invention relates to the field of defect detection of natural leather images, and in particular to a natural leather defect segmentation method based on an MLP multi-layer neural network. Background Art

[0002] Natural leather is a common raw material for daily necessities such as clothing, bags, and decorative accessories. However, natural leather has some defects to a greater or lesser extent. According to statistical results, there are eight types of the most common defects in leather. When using leather to manufacture products, these defects should be avoided as much as possible; and because the cost of natural leather is relatively high, low-cost consumption should be ensured as much as possible. In addition to reducing waste during production, high-precision detection and segmentation should be achieved as much as possible when detecting defects.

[0003] Machine learning technology has made great progress in recent years, which provides new ideas for image segmentation. In the past, image segmentation mostly used traditional segmentation methods, which often resulted in poor image segmentation results due to objective conditions such as lighting. Among the existing technologies, machine learning methods are particularly common and have great advantages in segmentation results. Deep learning technology has the strongest development momentum, but its demand for samples is too high and its requirements for software and hardware are also high. Therefore, we propose to combine MLP multi-layer neural network and traditional image processing methods for natural leather defect segmentation.

[0004] The present invention develops a natural leather defect detection method based on machine learning technology, enhances image quality through traditional image processing technology, and then uses MLP multi-layer neural network to improve the accuracy of defect detection and segmentation accuracy. The present invention can train a network model with high segmentation accuracy using a small sample size and can achieve real-time prediction. Summary of the invention

[0005] The purpose of the present invention is to provide a natural leather defect detection method with high detection accuracy and high segmentation precision in order to solve the above problems. The method can quickly obtain the segmentation results of the image and save the outer contour and defect contour results of the leather in the form of a vector diagram. The method has high working reliability and fast prediction speed.

[0006] To achieve the above-mentioned object of the invention, the present invention provides a natural leather defect segmentation method based on MLP multi-layer neural network, comprising the following steps:

[0007] S1: Collect leather images through the image acquisition module, and divide the collected leather images into a training set and a test set;

[0008] S2: Mark the leather image according to the different leather colors to generate a leather color feature map;

[0009] S3: Create an MLP multi-layer neural network classifier;

[0010] S4: converting the leather color feature map into a feature vector as the input of the MLP multi-layer neural network, training it using the training set, and saving the training result as a model file;

[0011] S5: perform image preprocessing on leather images in the test set;

[0012] S6: Use the model file to predict the preprocessed leather image and output the defect segmentation result of the leather image;

[0013] S7: Use traditional image processing methods to enrich the details of the defect segmentation image to obtain a complete leather segmentation image;

[0014] S8: Extract the defects of the leather segmentation image and the edge position information of the leather contour and convert the coordinates into a DXF file.

[0015] Furthermore, in the above step S3, the activation function of the hidden layer of the MLP multi-layer neural network adopts the hyperbolic tangent function tanh(x), and its calculation formula is as follows:

[0016]

[0017] In the formula, x is the input, which is the matrix of feature vectors in the neural network;

[0018] The value of the output layer of the MLP multi-layer neural network classifier is converted to a value range of 0 to 1. This conversion function is called the softmax function, and its calculation formula is:

[0019]

[0020]

[0021] Among them, y k represents the output value of the kth neuron, represents the output value of the kth neuron in the second layer (hidden layer), and They represent the weight and bias of the kth neuron in the second layer respectively.

[0022] Furthermore, the above step S4 uses the training set for training and needs to update the weights according to the cost function J(ω, b), which is characterized in that a regularization term is added to the cost function, and the formula of the updated cost function is as follows:

[0023]

[0024] In the formula, m represents the total number of neurons, y (i)is the true output value of the i-th layer, is the predicted value of the i-th layer, is the penalty term for regularization, λ is the regularization parameter, l is the number of layers in the current neural network, L is the total number of layers in the neural network, and θ i is the regularization function value of the lth layer.

[0025] Furthermore, the image preprocessing steps in the above step S5 are as follows:

[0026] S5.1: Separate the leather image into three sub-channels: R channel, G channel, and B channel;

[0027] S5.2: Filter and image equalize the three sub-channels respectively;

[0028] S5.3: merge the three sub-channels;

[0029] S5.4: performing histogram transformation on the merged image to enhance the contrast of the image;

[0030] S5.5: performing filtering processing on the contrast-enhanced image;

[0031] S5.6: Obtaining the processed leather image.

[0032] Furthermore, in the above step S6, the model file is used to predict the leather images in the test set to obtain the defect segmentation results, and the average prediction time of the leather images and the average values ​​of the quantitative evaluation indicators of the defect segmentation result images and the manually labeled images are calculated, wherein the quantitative evaluation indicators include precision, recall, and false positive rate.

[0033] Furthermore, the steps of traditional image processing in the above step S7 are:

[0034] S7.1: Perform OTSU threshold segmentation on the original leather image to obtain the leather ROI area;

[0035] S7.2: Using the ROI region as a mask, use basic image operations (AND operations) to obtain the entire leather region of the segmented image;

[0036] S7.3: Perform morphological closing operation on the entire leather area to enrich defect details.

[0037] The present invention trains the neural network through the color feature map, which requires only a small amount of training data, thereby reducing the cost of collecting training images, and has the characteristics of fast training speed and high accuracy. The present invention combines the traditional image processing method with the machine learning method, enriches the details of the segmentation result, makes the segmentation result more accurate, and lays the foundation for the subsequent leather layout and cutting process. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the overall flow chart of defect detection and segmentation of the present invention;

[0039] Figure 2 A schematic diagram of the structure of an image acquisition module according to a preferred embodiment of the present invention;

[0040] Figure 3 A schematic diagram of a color characteristic diagram of a preferred embodiment of the present invention;

[0041] Figure 4 A schematic diagram of the MLP network structure of a preferred embodiment of the present invention;

[0042] Figure 5 A schematic diagram of preprocessing of a channel separation operation according to a preferred embodiment of the present invention;

[0043] Figure 6 It is an MLP defect segmentation prediction diagram of a preferred embodiment of the present invention;

[0044] Figure 7 A complete defect segmentation map after morphological enrichment according to a preferred embodiment of the present invention;

[0045] Figure 8 This is an edge detection diagram of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention;

[0047] See also Figure 1 The natural leather defect segmentation method based on the MLP multi-layer neural network provided by the present invention comprises the following steps:

[0048] S1: Collect leather images through the image acquisition module, and divide the collected leather images into a training set and a test set;

[0049] S2: Mark the leather image according to the different leather colors to generate a leather color feature map;

[0050] S3: Create an MLP multi-layer neural network classifier;

[0051] S4: converting the leather color feature map into a feature vector as the input of the MLP multi-layer neural network, training it using the training set, and saving the training result as a model file;

[0052] S5: perform image preprocessing on leather images in the test set;

[0053] S6: Use the model file to predict the preprocessed leather image and output the defect segmentation result of the leather image;

[0054] S7: Use traditional image processing methods to enrich the details of the defect segmentation image to obtain a complete leather segmentation image;

[0055] S8: Extract the defects of the leather segmentation image and the edge position information of the leather contour and convert the coordinates into a DXF file.

[0056] As a preferred embodiment of the present invention, Figure 2 As shown, the image acquisition module of the present invention includes: scanning the leather with an industrial camera to obtain a complete leather image, transmitting it to an industrial computer through an image acquisition card, generating a data set, and dividing the data set into a test set and a training set in a ratio of 1:9.

[0057] As a preferred embodiment of the present invention, the image marking in step S2 includes: marking the images in the training set folder according to the color difference, and artificially creating a color feature map. The color feature map of a preferred embodiment of the present invention is shown in FIG. Figure 3 .

[0058] As a preferred embodiment of the present invention, see Figure 4 ,The MLP multilayer neural network classifier in step S3 is a multilayer perceptron neural network used for classification, which consists of an input layer, an output layer and a hidden layer;

[0059] As a preferred embodiment of the present invention, the activation function of the hidden layer of the MLP multi-layer neural network in step S3 adopts the hyperbolic tangent function tanh(x), and its calculation formula is as follows:

[0060]

[0061] In the formula, x is the input, which is the matrix of feature vectors in the neural network;

[0062] In this embodiment, the activation function formula used in the hidden layer is:

[0063]

[0064] in, is the output of the first layer, z j is the function value of the activation function, j is the number of the neuron;

[0065] The calculation formula is as follows:

[0066]

[0067] In the formula, x i is the input data, matrix and vector are the weights and biases of the MLP input layer respectively;

[0068] As a preferred embodiment of the present invention, the value of the output layer of the MLP multi-layer neural network classifier in step S3 is converted to a value range of 0 to 1. This conversion function is called a softmax function, and its calculation formula is:

[0069]

[0070]

[0071] Among them, y k represents the output value of the kth neuron, represents the output value of the kth neuron in the second layer (hidden layer), and Represent the weight and bias of the kth neuron in the second layer respectively;

[0072] As a preferred embodiment of the present invention, the parameter preprocessing of the MLP multi-layer neural network classifier in step S3 adopts principal component analysis PCA;

[0073] As a preferred embodiment of the present invention, the network training step in step S4 includes: converting the color feature map into a feature vector, and passing it into the MLP multi-layer neural network for training;

[0074] The process of training the model is the process of optimizing the cost function. The present invention adds a regularization term to the cost function of the neural network to prevent overfitting.

[0075] The network training in step S4 needs to update the weights according to the cost function J(ω, b). The cost function is a function of the weight ω and the offset b. The optimal values ​​of ω and b that can minimize the cost function (close to 0) are calculated through iteration. The formula of the cost function is as follows:

[0076]

[0077] In the formula, m represents the total number of neurons, y (i) is the output value of the i-th layer, is the predicted value of the i-th layer;

[0078] In order to avoid overfitting, a regularization term is added to the cost function of the neural network, so the formula of the updated cost function is as follows:

[0079]

[0080] In the formula, is the penalty term used for regularization, where λ is the regularization parameter, l is the number of layers of the current neural network, L is the total number of layers of the neural network, and θ i is the regularization function value of the lth layer;

[0081] The MLP network structure is built and trained. The prediction diagram of a preferred embodiment of the present invention is shown in Figure 5 ;

[0082] The image preprocessing steps in step S5 are as follows:

[0083] S5.1: Separate the image into R channel, G channel, and B channel;

[0084] S5.2: Perform filtering and image equalization processing to different degrees on the three sub-channels;

[0085] S5.3: merge the three sub-channels;

[0086] S5.4: performing histogram transformation on the merged image to enhance the contrast of the image;

[0087] S5.5: performing filtering processing on the contrast-enhanced image;

[0088] S5.6: obtaining a processed leather image;

[0089] See also Figure 6 , shows the effect diagram of filtering and image equalization processing of three sub-channels;

[0090] In step S6, the model file is used to predict the leather images in the test set to obtain defect segmentation results, and the average prediction time of the leather images and the average values ​​of quantitative evaluation indicators of the defect segmentation result images and the manually marked images are calculated, wherein the quantitative evaluation indicators include precision rate, recall rate, and false positive rate. According to the analysis of the results of the test set, the average calculation time of the model of a preferred embodiment of the present invention is 706.7ms, the precision rate and recall rate can reach more than 98.3%, and the false positive rate is 1.7%.

[0091] Figure 5 The defects in the segmented image shown are broken, and the segmentation is not complete. As a preferred embodiment of the present invention, the steps of traditional image processing in step S7 are:

[0092] S7.1: Perform OTSU threshold segmentation on the original leather image to obtain the leather ROI area;

[0093] S7.2: Using the ROI region as a mask, use basic image operations (AND operations) to obtain the entire leather region of the segmented image;

[0094] S7.3: Perform morphological closing operations on the entire leather area to enrich defect details, see Figure 7 ;

[0095] The effect of edge detection of leather outer contour and defects, see Figure 8 ;

[0096] The steps of converting into a DXF file in step S8 are:

[0097] S8.1: Organize the coordinate information of the txt file into an Excel table;

[0098] S8.2: Edit these coordinates in the Excel spreadsheet using AutoLISP language commands;

[0099] S8.3: Copy the Excel spreadsheet contents into Notepad and save the file with the .LSP extension;

[0100] S8.4: Batch input into AutoCAD, generate vector graphics, and save as DXF files.

Claims

1. A natural leather defect segmentation method based on MLP multi-layer neural network, characterized in that: The following steps are involved: S1: Collect leather images through the image acquisition module, and divide the collected leather images into a training set and a test set; S2: Mark the leather image according to the different leather colors to generate a leather color feature map; S3: Create an MLP multi-layer neural network classifier; S4: converting the leather color feature map into a feature vector as the input of the MLP multi-layer neural network, training it using the training set, and saving the training result as a model file; S5: perform image preprocessing on leather images in the test set; S6: Use the model file to predict the leather images in the test set and output the defect segmentation results of the leather images; S7: Use traditional image processing methods to enrich the details of the defect segmentation image to obtain a complete leather segmentation image; S8: extracting the defects of the leather segmentation image and the edge position information of the leather contour and converting the coordinates into a DXF file; The activation function of the hidden layer of the MLP multi-layer neural network classifier in step S3 adopts the hyperbolic tangent function tanh(x), and its calculation formula is as follows: In the formula, x is the input, which is the matrix of feature vectors in the neural network; The value of the output layer of the MLP multi-layer neural network classifier is converted to a value range of 0 to 1. This conversion function is called the softmax function, and its calculation formula is: Among them, y k represents the output value of the kth neuron, represents the output value of the kth neuron in the second layer, and Respectively represent the weight and bias of the kth neuron in the second layer; The parameter preprocessing of the MLP multi-layer neural network classifier adopts principal component analysis PCA; The step S4 uses the training set for training and needs to update the weights according to the cost function J(ω,b), which is characterized in that a regularization term is added to the cost function, and the formula of the updated cost function is as follows: In the formula, m represents the total number of neurons, y (i) is the true output value of the i-th layer, is the predicted value of the i-th layer, is the penalty term for regularization, λ is the regularization parameter, l is the number of layers in the current neural network, L is the total number of layers in the neural network, and θ l is the regularization function value of the lth layer.

2. The natural leather defect segmentation method based on MLP multi-layer neural network according to claim 1 is characterized in that: The image preprocessing steps in step S5 are as follows: S5.1: Separate the leather image into three sub-channels: R channel, G channel, and B channel; S5.2: Filter and image equalize the three sub-channels respectively; S5.3: merge the three sub-channels; S5.4: performing histogram transformation on the merged image to enhance the contrast of the image; S5.5: performing filtering processing on the contrast-enhanced image; S5.6: Obtaining the processed leather image.

3. The natural leather defect segmentation method based on MLP multi-layer neural network according to claim 1 is characterized in that: In step S6, the model file is used to predict the leather images in the test set to obtain defect segmentation results, and the average prediction time of the leather images and the average values ​​of quantitative evaluation indicators of the defect segmentation result images and the manually marked images are calculated, wherein the quantitative evaluation indicators include precision, recall, and false positive rate.

4. The natural leather defect segmentation method based on MLP multi-layer neural network according to claim 1 is characterized in that: The steps of conventional image processing in step S7 are: S7.1: Perform OTSU threshold segmentation on the original leather image to obtain the leather ROI area; S7.2: Using the ROI region as a mask, use basic image operations to obtain the entire leather region of the segmented image; S7.3: Perform morphological closing operation on the entire leather area to enrich defect details.

Images