Few-sample bamboo material defect detection method based on image recognition

Abstract

The invention discloses a few-sample bamboo material defect detection method based on image recognition. The problems of insufficient accuracy, insufficient sample quantity and the like in industrial production and processing are mainly solved. The invention provides a few-sample bamboo material defect detection method based on image recognition, which comprises the following steps of: positioning a preprocessed picture part by using Resnext, and further performing defect detection on a data set taking a few-negative-sample bamboo material as an example on the basis of improvement of a Resnet + Siamase network structure.

Description

Technical Field

[0001] The invention relates to the field of bamboo and wood materials, and in particular to a bamboo and wood material defect detection method based on image recognition with a small number of samples. Background Art

[0002] In modern manufacturing, bamboo and wood materials are widely used in furniture, construction, handicrafts and other fields due to their unique physical and aesthetic properties. However, due to the natural characteristics of bamboo and wood materials, they often produce various defects during the growth process, such as scars, cracks, wormholes, etc. These defects will directly affect the quality and service life of the finished product. Therefore, how to effectively detect defects in bamboo and wood materials, especially to achieve efficient and accurate defect detection under the condition of few samples, has become a difficult problem to be solved urgently.

[0003] There are many types of defects in bamboo and wood materials, mainly including the following categories:

[0004] Scarring: Cracks caused by internal stress, external environment or mechanical action during the growth, drying or processing of bamboo and wood. Cracks are one of the common defects in bamboo and wood materials, affecting the strength, appearance and service life of the material.

[0005] Cracks: Cracks usually appear on the surface or inside of bamboo and wood materials, which are caused by stress concentration or uneven drying. The presence of cracks will not only reduce the mechanical strength of the material, but also affect its durability.

[0006] Insect holes: Insect holes are caused by insects, which usually affect the appearance and strength of the material. Severe insect holes may also make the material lose its use value.

[0007] Discoloration: Discoloration defects in bamboo and wood materials are usually caused by environmental factors (such as sunlight, humidity, etc.) or chemical reactions. This defect mainly affects the appearance of the material, but in some cases it may also affect its structural performance.

[0008] Traditional defect detection of bamboo and wood materials mainly relies on manual visual inspection and empirical judgment. Although this method is simple, it has low accuracy and efficiency and is easily affected by subjective factors. In addition, due to the natural unevenness and complexity of bamboo and wood materials, manual inspection is difficult to ensure consistency. Therefore, there is an urgent need for an automated and intelligent defect detection method to solve this problem.

[0009] Defect detection method based on image recognition: With the rapid development of computer vision technology, defect detection method based on image recognition has gradually become an important means of bamboo and wood material detection. By analyzing and processing the surface images of bamboo and wood materials, various defects can be effectively identified and located. This method has the following significant advantages:

[0010] Non-contact detection: Image-based detection methods do not require direct contact with the material, avoiding damage to the material.

[0011] Efficiency and automation: Computer vision systems can realize automated detection, greatly improving detection efficiency and making them suitable for large-scale production.

[0012] High precision: Image processing algorithms can be used to achieve high-precision detection of tiny defects, reducing missed detection rates and false detection rates.

[0013] However, traditional image recognition algorithms usually rely on a large amount of labeled data for model training. In practical applications, it is time-consuming and laborious to obtain a large number of bamboo and wood material defect images and label them, especially for some rare or complex defects, the number of samples is very limited. Therefore, how to achieve efficient and accurate defect detection under the condition of few samples has become the focus of research. Summary of the invention

[0014] In order to overcome the shortcomings of the background technology, the present invention provides a bamboo and wood material defect detection method based on image recognition with a small number of samples, which mainly solves the problems of insufficient accuracy and insufficient sample quantity in industrial production and processing.

[0015] The technical solution of the present invention is:

[0016] A method for detecting defects in bamboo and wood materials based on image recognition using a small number of samples comprises the following steps:

[0017] Step 1: Collect bamboo and wood material surface images for training the Resnet+Siamase network; and perform data enhancement and preprocessing on the collected bamboo and wood material surface images;

[0018] Step 2: Locate the defect location;

[0019] Step 3: Defect detection: Resnet algorithm combined with Siamase network is used. The specific steps are as follows:

[0020] The input data is X1, X2, and X3, where X1 and X2 are samples of the same type, while X1 and X3 are samples of different types. The model Gw acts on the input samples, and Gw(X) converts the input samples X into a set of vectors. The distance Ew(X1, X2) is used to describe the distance between the output vectors of two input samples after being processed by the model. The loss function is as follows:

[0021]

[0022] L(W,(Y,X1,X2) i )=(1-Y)L G (E w (X1,X2)i )+YL I (E w (X1,X2) i )

[0023] When using contrast Loss, the above loss function changes to:

[0024]

[0025] ResNet was introduced. The core modules of ResNet are BottleNeck1 (BTNK1 for short) and BottleNeck2 (BTNK2 for short). The network structure of ResNet was introduced into the core layer of the Siamese network, and different numbers of 3x3 convolutional layers were added to BTNK1 and BTNK2.

[0026] In step 2, the ResNeXt algorithm is used to locate the defect position.

[0027] When BTNK2 remains unchanged, the number of convolutional layers of BTNK1 is 3.

[0028] When BTNK1 remains unchanged, the number of convolutional layers of BTNK2 is 3 or 2.

[0029] BTNK1 has 2 layers and BTNK2 has 3 layers.

[0030] In step 1, the Wiener filter is used to restore the clear boundary of the image, and its mathematical expression is:

[0031]

[0032] In the formula, H(u,v) represents the degradation transfer function, G(u,v) is the Fourier transform result of the blurred image, and SNR(u,v) represents the signal-to-noise ratio of the image.

[0033] The beneficial effects of the present invention are as follows: the present invention provides a bamboo and wood material defect detection method based on image recognition with a small number of samples, uses Resnext to locate the preprocessed image components, and further improves the network structure based on Resnet+Siamase to perform defect detection on a data set of bamboo and wood materials with a small number of negative samples as an example. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a logic block diagram of the present invention.

[0035] Figure 2 The comparison of the effects before and after Wiener filtering is shown, and the results show that this method performs well in removing motion blur.

[0036] Figure 3 A graphic showing the effect of using the ResNeXt algorithm to locate bamboo wood defects.

[0037] Figure 4 Schematic diagram of a twin neural network.

[0038] Figure 5 Schematic diagram of the ResNet network structure.

[0039] Figure 6 Schematic diagram of BTNK1 and BTNK2.

[0040] Figure 7 Schematic diagram of positive samples and negative samples. DETAILED DESCRIPTION

[0041] The present invention is further described below in conjunction with the accompanying drawings: As shown in the figure, a method for detecting defects in bamboo and wood materials based on image recognition using a small number of samples comprises the following steps:

[0042] Step 1: Collect bamboo and wood material surface images for training the Resnet+Siamase network; and perform data enhancement and preprocessing on the collected bamboo and wood material surface images;

[0043] Step 2: Locate the defect position; Step 3: Defect detection; Use the Resnet algorithm combined with the Siamase network. The specific steps are as follows:

[0044] The input data is X1, X2, and X3, where X1 and X2 are samples of the same type, while X1 and X3 are samples of different types. The model Gw (where w is a model parameter) acts on the input sample, and Gw(X) converts the input sample X into a set of vectors. The distance Ew(X1, X2) is used to describe the distance between the output vectors of two input samples after being processed by the model. The loss function is as follows:

[0045]

[0046] L(W,(Y,X1,X2) i )=(1-Y)L G (E w (X1,X2) i )+YL I (E w (X1,X2) i )

[0047] When using contrast Loss, the above loss function changes to:

[0048]

[0049] ResNet is introduced. The core modules of ResNet are BottleNeck1 (BTNK1 for short) and BottleNeck2 (BTNK2 for short). The network structure of ResNet (which can be ResNet50) is introduced into the core layer of the Siamese network, and different numbers of 3x3 convolutional layers are added to BTNK1 and BTNK2.

[0050] In step two, the ResNeXt algorithm is used to locate the defect position. The ResNeXt algorithm is a more mature choice. By introducing group convolution in the residual network, ResNeXt effectively enhances the feature extraction capability and performs well in image recognition, object detection and positioning tasks. Its modular design makes the network easy to expand and optimize. Group convolution divides the input channel into multiple groups, each group performs a convolution operation independently, and then merges the results into the output channel. In this way, ResNeXt can increase the complexity of the model by increasing the number of groups (cardinality) without increasing the width and depth of the network. How to train the model through existing bamboo and wood defect pictures is a prior art and will not be repeated here. The main feature of this application is how to accurately locate with few samples.

[0051] When BTNK2 remains unchanged, the number of convolutional layers of BTNK1 is 3.

[0052] When BTNK1 remains unchanged, the number of convolutional layers of BTNK2 is 3 or 2.

[0053] BTNK1 has 2 layers and BTNK2 has 3 layers.

[0054] In step 1, the Wiener filter is used to restore the clear boundary of the image, and its mathematical expression is:

[0055]

[0056] In the formula, H(u,v) represents the degradation transfer function, G(u,v) is the Fourier transform result of the blurred image, and SNR(u,v) represents the signal-to-noise ratio of the image.

[0057] Firstly, the surface images of bamboo and wood materials are collected for training the Resnet+Siamase network; then, data enhancement and preprocessing are performed on the collected surface images of bamboo and wood materials.

[0058] Due to the influence of factors such as ambient light, speed, camera angle, aperture setting and color difference, it may not be possible to ensure that all images can clearly capture potential defective parts during data acquisition. Therefore, it is very important to select a suitable filter to restore the clear boundary of the image as much as possible. Among many filter schemes, the present invention adopts the Wiener filter. The basic principle is to find a filter to minimize the mean square error between the repaired image and the original image, so as to restore the clear outline of the original image as much as possible. Its mathematical expression is:

[0059]

[0060] In the formula, H(u,v) represents the degradation transfer function, G(u,v) is the Fourier transform result of the blurred image, and SNR(u,v) represents the signal-to-noise ratio of the image. This paper uses Gaussian filtering for noise reduction and uses the comparison of the before and after images to estimate the signal-to-noise ratio. The application of denoising filters helps to avoid the appearance of ringing effects. Figure 2 The comparison of the effects before and after Wiener filtering is shown, and the results show that this method performs well in removing motion blur.

[0061] Before performing defect detection on bamboo and wood materials, it is first necessary to accurately locate the defect position in the original image. Among the many object detection algorithms, ResNeXt is a more mature choice. By introducing grouped convolution in the residual network, ResNeXt effectively enhances the feature extraction capability and performs well in image recognition, object detection and positioning tasks. Its modular design makes the network easy to expand and optimize. Grouped convolution divides the input channel into multiple groups, each of which performs a convolution operation independently, and then merges the results into the output channel. In this way, ResNeXt can increase the complexity of the model by increasing the number of groups (cardinality) without increasing the width and depth of the network. Figure 3 The effect of using the ResNeXt algorithm to locate bamboo wood defects is demonstrated.

[0062] After locating the defects in the image, the next step is to perform defect detection. An important issue is sample imbalance, especially the large number of positive samples and the small number of negative samples. How to effectively solve the scarcity of samples becomes a key challenge for defect detection. One way is to solve it through data accumulation, but this usually requires a lot of manpower, material resources and time, especially when there are more inspection items, more data is needed. Another way is to approach from the perspective of the algorithm to fundamentally solve the problem of sample imbalance. In fact, humans do not have high requirements for the number of negative samples when identifying defects, so a good defect detection algorithm should be theoretically close to human recognition ability. The present invention adopts a twin neural network (Siamase) as an effective solution.

[0063] like Figure 4 As shown in the figure, the input data is X1, X2, and X3, where X1 and X2 are samples of the same type, while X1 and X3 are samples of different types. The model Gw (where w is a model parameter) acts on the input sample, and Gw(X) converts the input sample X into a set of vectors. The distance Ew(X1, X2) is used to describe the distance between the output vectors of two input samples after being processed by the model. The loss function is as follows:

[0064]

[0065] L(W,(Y,X1,X2) i )=(1-Y)L G (E w (X1,X2) i )+YL I (E w (X1,X2) i )

[0066] When using contrast Loss, the above loss function changes to:

[0067]

[0068] According to our definition of the distance between two vectors: the vector distance between samples of the same type should be smaller than the vector distance between samples of different types, and the smaller the distance, the higher the probability that the samples belong to the same category.

[0069] Traditional convolutional neural networks have a disadvantage, that is, when the network depth reaches a certain level, the accuracy of the model will first reach saturation and then drop rapidly. This is because during the back propagation process, if the gradient of each layer is between (0,1), as the number of layers increases, the gradient will gradually decrease, and eventually the gradient will disappear; on the contrary, if the gradient is greater than 1, the layer-by-layer amplification will cause the gradient to explode. Therefore, simply increasing the number of layers will inevitably lead to network degradation problems. These two phenomena are called gradient explosion and gradient disappearance, respectively. The introduction of ResNet, such as Figure 5 The ResNet network structure shown in the figure can learn more detailed features during the back propagation process by adding identity mapping, thereby improving the accuracy of the network.

[0070] The core modules of ResNet are BottleNeck1 (BTNK1 for short) and BottleNeck2 (BTNK2 for short). The purpose is to introduce a residual network structure so that as the network depth increases, the accuracy of the deep network will not be lower than the accuracy of the optimal shallow network. The design of ResNet ensures that the performance of the deep network will not decrease due to network degradation when the number of layers increases.

[0071] In this paper, the network structure of ResNet50 is introduced into the core layer of Siamese network, and its structure is improved by adding different numbers of 3x3 convolutional layers in BTNK1 and BTNK2 to solve the problem of bamboo and wood material defect detection based on a small number of negative samples, such as Figure 6 .

[0072] Improvements of BTNK1 and BTNK2: Recall, also known as sensitivity or recall, is an important indicator for measuring the ability of a classification model to identify positive samples. It indicates the proportion of positive samples correctly identified by the model among all samples that are actually positive.

[0073] The calculation formula for recall is:

[0074] Recall = True Positives (TP) / (True Positives (TP) + False Negatives (FN))

[0075] True Positive (TP): The number of samples correctly predicted as positive by the model. False Negative (FN): The number of samples that are actually positive but incorrectly predicted as negative by the model. The database of bamboo and wood material discoloration (including 1027 positive samples and 32 negative samples, such as Figure 7 shown) were used for testing.

[0076] First, while keeping BTNK2 unchanged, different numbers of 3x3 convolutional layers were added to BTNK1 for training. The experimental results are shown in Table 1(a). The recall rate gradually increased from 0.812 with the increase of the number of convolutional layers, reaching the highest value of 0.923 when the number of convolutional layers was 3, and then dropped to the lowest value of 0.856 as the number of layers continued to increase; while the accuracy rate also gradually increased from 0.856 with the increase of the number of layers, reaching the highest value of 0.929 at 3 layers, and then dropped to the lowest value of 0.905 as the number of layers increased. The experimental results show that both the accuracy and recall rate reached the highest value at 3 layers, and the recall rate dropped faster as the number of layers increased.

[0077] Improvement of BTNK2: Next, BTNK1 is kept unchanged and different numbers of 3x3 convolutional layers are added to BTNK2 for training. The experimental results are shown in Table 1(b). The recall rate gradually increases from 0.878 with the increase of the number of convolutional layers, reaching the highest value of 0.922 at 2 layers, and then drops to the lowest value of 0.857 with the increase of the number of layers; while the accuracy rate reaches the highest value of 0.925 when the number of layers increases to 3 layers, and then drops to the lowest value of 0.889 with the continued increase of the number of layers. The experimental results show that the accuracy and recall rate reach the highest values ​​at the 2nd and 3rd layers respectively.

[0078]

[0079] Table 1(a) shows the recall and precision obtained by adding 1 to 5 layers of 3x3 convolutional networks to BTNK1 while keeping BTNK2 unchanged.

[0080] Table 1(b) shows the recall and accuracy obtained by adding 1 to 5 layers of 3x3 convolutional networks to BTNK2 while keeping BTNK1 unchanged.

[0081] Combined improvement of BTNK1 and BTNK2: According to the experimental results of 3.1.1 and 3.1.2 above, 2 to 4 layers of 3x3 convolutional network layers are added to BTNK1, and the structure of BTNK1 is kept unchanged. At the same time, 1 to 3 layers of 3x3 convolutional network layers are added to BTNK2 for combined training. The experimental results show that the recall rate and accuracy of this combination are higher than those of improving BTNK1 or BTNK2 alone. When BTNK1 is set to 2 layers and BTNK2 is set to 3 layers, the highest accuracy rate of 0.942 and recall rate of 0.939 are achieved.

[0082] The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention. The embodiments should not be regarded as limiting the present invention, but any improvements made based on the spirit of the present invention should be within the scope of protection of the present invention.

Claims

1. A method for detecting defects in bamboo and wood materials based on image recognition with few samples, characterized in that: The following steps are included: Step 1: Collect bamboo and wood material surface images for training the Resnet+Siamase network; and perform data enhancement and preprocessing on the collected bamboo and wood material surface images; Step 2: Locate the defect location; Step 3: Defect detection: Resnet algorithm combined with Siamase network is used. The specific steps are as follows: The input data is X1, X2, and X3, where X1 and X2 are samples of the same type, while X1 and X3 are samples of different types. The model Gw acts on the input samples, and Gw(X) converts the input samples X into a set of vectors. The distance Ew(X1, X2) is used to describe the distance between the output vectors of two input samples after being processed by the model. The loss function is as follows: L(W,(Y,X1,X2) i )=(1-Y)L G (E w (X1,X2) i )+YL I (E w (X1,X2) i ) When using contrast Loss, the above loss function changes to: ResNet was introduced. The core modules of ResNet are BTNK1 and BTNK2. The network structure of ResNet was introduced into the core layer of the Siamese network, and different numbers of 3x3 convolutional layers were added to BTNK1 and BTNK2.

2. The method for detecting defects in bamboo and wood materials based on image recognition with a small number of samples according to claim 1, characterized in that: In step 2, the ResNeXt algorithm is used to locate the defect position.

3. The method for detecting defects in bamboo and wood materials based on image recognition with a small number of samples according to claim 2, characterized in that: When BTNK2 remains unchanged, the number of convolutional layers of BTNK1 is 3.

4. The method for detecting defects in bamboo and wood materials based on image recognition with a small number of samples according to claim 3 is characterized by: When BTNK1 remains unchanged, the number of convolutional layers of BTNK2 is 3 or 2.

5. The method for detecting defects in bamboo and wood materials based on image recognition with a small number of samples according to claim 4, characterized in that: BTNK1 has 2 layers and BTNK2 has 3 layers.

6. The method for detecting defects in bamboo and wood materials based on image recognition with a small number of samples according to claim 5, characterized in that: In step 1, the Wiener filter is used to restore the clear boundary of the image, and its mathematical expression is: In the formula, H(u,v) represents the degradation transfer function, G(u,v) is the Fourier transform result of the blurred image, and SNR(u,v) represents the signal-to-noise ratio of the image.

Images