A method, system and device for defect detection of furniture panels

Abstract

The present invention discloses a defect detection method, system, and device for furniture panels. The method comprises: acquiring a sample image and annotating defects in the sample image to obtain defect labels, then constructing a sample data set and slicing and randomly dividing the sample data set to obtain a training data set and a test data set; performing data preprocessing on the training data set and training a pre-constructed detector based on the preprocessed training data set to obtain a defect detection model; and performing feature extraction, classification, regression processing, and non-maximum suppression processing on the test data set in sequence based on the defect detection model to obtain defect detection results. The system comprises: an image acquisition module, a preprocessing module, a defect detection model training module, and a defect detection module. By using the present invention, it is possible to accurately locate and classify surface defects of panels containing complex textures under limited sampling conditions.

Description

Technical Field

[0001] The present invention relates to the field of machine vision defect detection, and in particular to a defect detection method, system and device for furniture panels. Background Art

[0002] During the production process of furniture panels, process problems such as chipped edges, hole and groove defects, and decorative flaws are often inevitable. These defects have a significant impact on the subsequent use of the furniture panels, and may cause serious problems such as inability to bind, affecting aesthetics, and reducing service life. Therefore, how to conduct efficient and accurate real-time defect detection for the diverse and large number of furniture panel products on the actual production line has become an important problem facing the furniture industry.

[0003] Existing defect detection algorithms can be roughly divided into two categories: traditional algorithms and learning-based algorithms. Most traditional algorithms rely on feature engineering based on prior knowledge, encompassing statistical, structural, spectroscopy, and model-based approaches. Learning-based algorithms can be further categorized into classic machine learning algorithms and deep learning algorithms. Machine learning utilizes mathematical algorithms to learn and analyze data, thereby making predictions and decisions about the test data. However, these classic methods generally require human assistance, have limited generalization capabilities, require instrument debugging before testing, and incur high equipment development costs, making them difficult to adapt to real-world industrial applications.

[0004] With the development of deep learning, due to its high precision and low cost, it has become the most popular automatic defect detection method in recent years. However, deep learning-based methods rely on large amounts of learning data to drive the model, update and improve the model's detection accuracy. However, the actual number of defect samples is extremely difficult to obtain. In addition, furniture panels also have interference factors such as the diverse scale of defects and complex surface textures. Therefore, existing models cannot achieve good recognition results and accuracy when directly applied to furniture panels. Summary of the Invention

[0005] In order to solve the above technical problems, the purpose of the present invention is to provide a defect detection method, system and device for furniture panels, which can accurately locate and classify surface defects of panels with complex textures under limited sampling conditions.

[0006] The first technical solution adopted by the present invention is: a defect detection method for furniture panels, comprising the following steps:

[0007] Obtain a sample image and mark defects in the sample image to obtain a defect label;

[0008] Construct a sample dataset based on sample images and defect labels, slice and randomly divide the sample dataset to obtain a training dataset and a test dataset;

[0009] Performing data preprocessing on the training data set to obtain a preprocessed training data set;

[0010] The pre-built detector is trained based on the pre-processed training dataset to obtain a defect detection model;

[0011] Based on the defect detection model, the test data set is sequentially subjected to feature extraction, classification, and regression processing to obtain defect candidate information;

[0012] The defect candidate information is screened based on the non-maximum suppression method to obtain the defect detection results.

[0013] Furthermore, the defect label includes the coordinates of the upper left corner and the lower right corner of the rectangular frame area where each defect is located in the image and the category of the defect.

[0014] Furthermore, the data preprocessing of the training data set includes data expansion and data enhancement.

[0015] Furthermore, the step of constructing a sample dataset based on the sample images and defect labels and slicing and randomly dividing the sample dataset to obtain a training dataset and a test dataset specifically includes:

[0016] Slicing the image data in the sample data set twice to obtain slice data of a first specification and slice data of a second specification;

[0017] The black edges of the slice data of the first specification are cropped, and the black edges of the slice data of the second specification are retained;

[0018] Merge the slice data and remove duplicate samples to obtain a sample group;

[0019] The sample group is randomly divided into training data sets and test data sets according to the preset ratio.

[0020] Furthermore, the training step of training the pre-built detector based on the pre-processed training data set specifically includes:

[0021] Build a detector based on the improved Retinanet network;

[0022] The preprocessed training data set is input into the detector for feature extraction to obtain low-level features and high-level features of the training data set;

[0023] The low-level features and high-level features are fused based on texture suppression to obtain fused features;

[0024] Input the fused features into the FPN network to construct a feature pyramid;

[0025] Input the features of each layer of the feature pyramid into the classification subnet and regression subnet respectively to detect defect targets of different sizes and obtain training detection information;

[0026] Compare the training detection information with the corresponding defect labels and adjust the detector model parameters.

[0027] Furthermore, the feature pyramid introduces an attention mechanism module to add weights to the features of different channels in the feature map.

[0028] Furthermore, the step of fusing low-level features and high-level features based on texture suppression to obtain fused features specifically includes:

[0029] The five residual blocks of the feature extraction network ResNet are used as the input of the feature fusion module;

[0030] Performing channel conversion on the output of the first residual block and the output of the second residual block based on convolution to obtain the first residual block and the second residual block having the same number of channels as the low-level features output by the fourth residual block and the fifth residual block;

[0031] Downsampling the first residual block output and the second residual block output respectively based on the ReLU activation function to obtain the downsampled first residual block output and the second residual block output;

[0032] Compare the downsampled first residual block output with the fourth residual block output on a feature point-by-feature basis and take their intersection to obtain a first intersection feature;

[0033] Compare the downsampled second residual block output with the fifth residual block output feature point by feature point and take the intersection to obtain a second intersection feature;

[0034] The output of the third residual block is used as the first feature, the first intersection feature and the output of the fourth residual block are added as the second feature, and the second intersection feature and the output of the fifth residual block are added as the third feature, which is input into the subsequent feature pyramid network.

[0035] The second technical solution adopted by the present invention is: a defect detection system for furniture panels, comprising:

[0036] An image acquisition module is used to sample surface defect samples of furniture panels and obtain two-dimensional images of the surface defects of the furniture panels;

[0037] A preprocessing module is used to construct a sample data set and preprocess the data in the sample data set to obtain a preprocessed sample data set;

[0038] The defect detection model training module trains the pre-built detector based on the pre-processed sample data set to obtain a weighted model for furniture board surface defect detection;

[0039] The defect detection module performs feature extraction, classification, regression and non-maximum suppression on the test data set based on the defect detection model to obtain the location and category information of the defects contained in the test sample.

[0040] The third technical solution adopted by the present invention is: a defect detection device for furniture panels, comprising:

[0041] at least one processor;

[0042] at least one memory for storing at least one program;

[0043] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned defect detection method for furniture panels.

[0044] The beneficial effects of the method and system of the present invention are as follows: the present invention first slices the collected large-scale furniture board images by slicing to obtain defect sample data sets of different scales, and then performs data enhancement and data expansion on the sample data sets, effectively solving the problems of small sample number and unbalanced sample categories. Finally, the pre-constructed single-stage defect detection model is trained based on the preprocessed training data sample set to obtain a defect detection model, which realizes accurate recognition of multi-scale and multi-category defects on the surface of furniture boards. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flowchart of the steps of a defect detection method for furniture panels according to the present invention;

[0046] Figure 2 This is a structural block diagram of a defect detection system for furniture panels according to the present invention;

[0047] Figure 3 3. It is a schematic diagram comparing defect classification data of the Retinanet model and the defect detection model in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0049] Reference Figure 1The present invention provides a method for detecting defects in furniture panels, the method comprising the following steps:

[0050] S1. Obtain a sample image and mark defects in the sample image to obtain a defect label;

[0051] Specifically, an industrial camera was used to capture two-dimensional images of the furniture board surface. The maximum resolution of the captured images was 5726*25266, and the minimum resolution was 1101*3061. An image annotation tool was then used to annotate the defects in the captured images with rectangular frames, obtaining the corresponding defect labels.

[0052] The defect label contains the coordinates of the upper left corner of the rectangular area where each defect is located in the image (x min ,y min ), the coordinate of the lower right corner (x max ,y max ) and the defect category N, N∈{1,2,3,...,7}, which means there are 7 types of defects in the dataset, and the corresponding defect names are scratches, black spots, edge damage, round hole damage, groove damage, corner damage and cracks.

[0053] S2. Construct a sample dataset based on the sample images and defect labels and slice and randomly divide the sample dataset to obtain a training dataset and a test dataset;

[0054] S2.1. Slice the image data in the sample data set twice to obtain slice data of a first specification and slice data of a second specification.

[0055] Specifically, the slice data of the first specification is 600*600, and the slice data of the second specification is 1024*1024;

[0056] S2.2. Crop the black edges of the slice data of the first specification, and retain the black edges of the slice data of the second specification;

[0057] S2.3. Merge the slice data and remove duplicate samples to obtain a sample group;

[0058] Specifically, the slice data are merged and some duplicate samples are removed to obtain 2490 groups of samples, each of which includes images and labels.

[0059] S2.4. Randomly divide the sample group according to the preset ratio to obtain the training data set and the test data set.

[0060] S3. Preprocess the training data set to obtain a preprocessed training data set;

[0061] Specifically, the training dataset is augmented by random horizontal flipping, vertical flipping, random shearing, random scaling, random length and width distortion, and random color gamut distortion.

[0062] The expanded data is enhanced using the Mosaic method, that is, four groups of data are randomly selected and randomly cropped, and then spliced ​​into a picture as a new training data group.

[0063] S4. Training the pre-built detector based on the pre-processed training data set to obtain a defect detection model;

[0064] S4.1. Build a detector based on the improved Retinanet network.

[0065] Specifically, improvements to the Retinanet network:

[0066] 1) Add a feature fusion module to the original feature extraction network to fuse high-level and low-level features to obtain fused features containing more detailed information;

[0067] 2) In the process of constructing the feature pyramid, an attention mechanism module is added to the output of the first two network layers of the pyramid, and then weights are added to the features of different channels in the feature map, thereby enhancing important features and weakening unimportant features, thereby enhancing the directionality of feature extraction.

[0068] Reference Figure 3 , showing the harmonic mean data of the precision and recall of defect classification of the Retinanet model and the defect detection model on the test set. The left bar chart of each category is the Retinanet model, and the right bar chart is the defect detection model. It can be seen that the defect detection model has significantly higher classification performance for various types of defects than the Retinanet model. From the test results, it can be seen that this method can efficiently detect multi-scale defects in large-resolution images with complex textures.

[0069] S4.2. Use the pre-trained model weights obtained by training on the ImageNet dataset as the initial weights of the detector to complete the initialization of the network model;

[0070] Specifically, during detector training, the weights of the pre-trained model are used for initialization. The entire network is trained using a first-order gradient optimization stochastic objective function algorithm (Adam), with a global learning rate set to 1e-4. During the iteration process, the StepLR mechanism is used to adjust the learning rate, with a multiplication factor of gamma = 0.96 set to update the learning rate after each epoch of training. The weight factors α and β in the Focal Loss function are set to 0.5 and 2, respectively. Training is divided into two phases: the freezing phase and the unfreezing phase. In the experiment, the parameters of the main part of the model are first frozen in the freezing phase, that is, the feature extraction network is not changed, and only the model is fine-tuned. Then, unfreezing training is performed. This experiment trained a total of 50 epochs, and the best model was obtained at epoch 50.

[0071] S4.3. Input the preprocessed training data set into the detector for feature extraction to obtain low-level features and high-level features of the training data set;

[0072] S4.4. Fuse low-level features and high-level features based on texture suppression to obtain fused features;

[0073] Specifically, the layer 0 of the feature extraction network ResNet is recorded as L0, and the outputs of the four residual blocks are recorded as L1, L2, L3, and L4, which serve as the input of the feature fusion module;

[0074] Use a convolution with a convolution kernel of 1*1 to convert the number of channels of L0 and L1 to be consistent with L3 and L4;

[0075] Add a Relu activation function to the above L0 and L1 respectively, and then downsample them to obtain L0_Down and L1_Down respectively;

[0076] Compare L0_Down and L3 feature points by feature point, and take their intersection as fuse_mask0;

[0077] Similarly, L1_Down is compared with L4 feature point by feature point, and their intersection is taken as fuse_mask1;

[0078] Finally, L2 is used as F1, fuse_mask0 and L3 are added as F2, and fuse_mask1 and L4 are added as F3, which are input into the subsequent feature pyramid network.

[0079] S4.5. Input the fused features into the FPN network to construct a feature pyramid;

[0080] Among them, FPN: Feature Pyramid Networks.

[0081] Specifically, the obtained fusion features are input into FPN, and through the top-down pathway and lateral pathway, a rich and multi-scale feature pyramid is obtained from the single resolution of the image.

[0082] S4.6. Input the features of each layer of the feature pyramid into the classification subnet and the regression subnet respectively to detect defect targets of different sizes and obtain training detection information;

[0083] S4.7. Compare the training detection information with the corresponding defect labels and adjust the detector model parameters;

[0084] S4.8. Use the trained detector as a defect detection model.

[0085] S5. Based on the defect detection model, feature extraction, classification, and regression processing are performed on the test data set in sequence to obtain defect candidate information;

[0086] Specifically, the defect candidate information includes the position of the defect position candidate box in the test data set and the category probability information of the defect in the candidate box;

[0087] S6. Screen the defect candidate information based on the non-maximum suppression method to obtain the defect detection result.

[0088] Specifically, the furniture board defect detection operation is as follows:

[0089] The furniture board samples to be tested are input into the defect detection model to extract the defect features in the test samples and obtain multi-scale feature maps. The above feature maps are then input into the classification subnet and regression subnet respectively to obtain the defect feature categories, their probabilities and location coordinates. Finally, this set of information is filtered through non-maximum suppression to obtain the final defect detection results.

[0090] As a further preferred embodiment of the present method, the step of inputting the above-mentioned feature map into the classification subnet and the regression subnet respectively to obtain the defect feature category, its probability and location coordinates, and finally filtering this set of information through non-maximum suppression to finally obtain the best detection result specifically includes:

[0091] In the classification subnet, four 3×3 convolutional layers are used, each followed by a ReLU layer, followed by a 3×3 convolutional layer with 8*9 channels (8 is the number of defect categories, 9 is the number of anchors), and finally a sigmoid activation function is used to obtain the probability value of each anchor belonging to each category;

[0092] In the regression subnet, the structure is similar to the classification subnet, except that the number of channels in the last layer is 4*9, and the bounding box coordinates corresponding to each anchor can be obtained.

[0093] Use a classification score threshold of 0.05 to filter out candidate targets with too low classification probability, and then perform NMS post-processing on the remaining candidate targets;

[0094] Sort all remaining candidate boxes by classification score from largest to smallest, and record the classification categories of all candidate boxes. Then, begin traversing, extracting all candidate objects of each category. First, extract the first object into the retained object set. Then, calculate the Intersection over Union (IoU) between the first object and the remaining objects. Discard any candidate objects with an IoU greater than a threshold of 0.5. Repeat this process until no candidate objects remain, and then save the optimal detection result.

[0095] Reference Figure 2 , a defect detection system for furniture panels, comprising:

[0096] An image acquisition module is used to sample surface defect samples of furniture panels and obtain two-dimensional images of the surface defects of the furniture panels;

[0097] A preprocessing module is used to construct a sample data set and preprocess the data in the sample data set to obtain a preprocessed sample data set;

[0098] The defect detection model training module trains the pre-built detector based on the pre-processed sample data set to obtain a weighted model for furniture board surface defect detection;

[0099] The defect detection module performs feature extraction, classification, regression and non-maximum suppression on the test data set based on the defect detection model to obtain the location and category information of the defects contained in the test sample.

[0100] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0101] The beneficial effects of the present invention specifically include:

[0102] 1) Based on the improved RetinaNet, accurate recognition of multi-scale and multi-category defects on the surface of furniture panels is achieved.

[0103] 2) By adding a new feature fusion module, the texture features of the plate are effectively suppressed, allowing the network to fully learn the relevant features of defects of different sizes on the plate surface.

[0104] 3) The attention mechanism is introduced into the feature pyramid to increase the inspection model's attention to difficult-to-distinguish samples, further improving the performance of the inspection model.

[0105] A defect detection device for furniture panels:

[0106] at least one processor;

[0107] at least one memory for storing at least one program;

[0108] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned defect detection method for furniture panels.

[0109] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0110] A storage medium storing processor-executable instructions, characterized in that the processor-executable instructions, when executed by the processor, are used to implement the above-mentioned defect detection method for furniture panels.

[0111] The contents of the above method embodiments are all applicable to the present storage medium embodiment. The functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0112] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for detecting defects in furniture panels, characterized in that: The following steps are involved: Obtain a sample image and mark defects in the sample image to obtain a defect label; Construct a sample dataset based on sample images and defect labels, slice and randomly divide the sample dataset to obtain a training dataset and a test dataset; Performing data preprocessing on the training data set to obtain a preprocessed training data set; The pre-built detector is trained based on the pre-processed training dataset to obtain a defect detection model; Based on the defect detection model, the test data set is sequentially subjected to feature extraction, classification, and regression processing to obtain defect candidate information; The defect candidate information is screened based on the non-maximum suppression method to obtain the defect detection result; The training step of training the pre-built detector based on the pre-processed training data set specifically includes: Build a detector based on the improved Retinanet network; The preprocessed training data set is input into the detector for feature extraction to obtain low-level features and high-level features of the training data set; The low-level features and high-level features are fused based on texture suppression to obtain fused features; Input the fused features into the FPN network to construct a feature pyramid; Input the features of each layer of the feature pyramid into the classification subnet and regression subnet respectively to detect defect targets of different sizes and obtain training detection information; Compare the training detection information with the corresponding defect labels and adjust the detector model parameters; The step of fusing low-level features and high-level features based on texture suppression to obtain fused features specifically includes: The five residual blocks of the feature extraction network ResNet are used as the input of the feature fusion module; Performing channel conversion on the output of the first residual block and the output of the second residual block based on convolution to obtain the first residual block and the second residual block having the same number of channels as the low-level features output by the fourth residual block and the fifth residual block; Downsampling the first residual block output and the second residual block output respectively based on the ReLU activation function to obtain the downsampled first residual block output and the second residual block output; Compare the downsampled first residual block output with the fourth residual block output feature point by feature point and take the intersection to obtain a first intersection feature; Compare the downsampled second residual block output with the fifth residual block output on a feature point-by-feature basis and take the intersection to obtain a second intersection feature; The output of the third residual block is used as the first feature, the first intersection feature and the output of the fourth residual block are added as the second feature, and the second intersection feature and the output of the fifth residual block are added as the third feature, which is input into the subsequent feature pyramid network.

2. A method for detecting defects in furniture panels according to claim 1, characterized in that: The defect label includes the coordinates of the upper left corner and the lower right corner of the rectangular frame area where each defect is located in the image and the category of the defect.

3. The method for detecting defects in furniture panels according to claim 1, characterized in that: The data preprocessing of the training data set includes data expansion and data enhancement.

4. The method for detecting defects in furniture panels according to claim 1, characterized in that: The step of constructing a sample dataset based on the sample images and defect labels and slicing and randomly dividing the sample dataset to obtain a training dataset and a test dataset specifically includes: Slicing the image data in the sample data set twice to obtain slice data of a first specification and slice data of a second specification; The black edges of the slice data of the first specification are cropped, and the black edges of the slice data of the second specification are retained; Merge the slice data and remove duplicate samples to obtain a sample group; The sample group is randomly divided into training data sets and test data sets according to the preset ratio.

5. The method for detecting defects in furniture panels according to claim 1, characterized in that: The feature pyramid introduces an attention mechanism module to add weights to the features of different channels in the feature map.

6. A defect detection system for furniture panels, characterized in that: include: An image acquisition module is used to sample surface defect samples of furniture panels and obtain two-dimensional images of the surface defects of the furniture panels; A preprocessing module is used to construct a sample data set and preprocess the data in the sample data set to obtain a preprocessed sample data set; The defect detection model training module trains the pre-built detector based on the pre-processed training data set to obtain a weighted model for furniture board surface defect detection; The defect detection module performs feature extraction, classification, regression, and non-maximum suppression on the test data set based on the defect detection model to obtain the location and category information of the defects contained in the test sample; The training step of training the pre-built detector based on the pre-processed training data set specifically includes: Build a detector based on the improved Retinanet network; The preprocessed training data set is input into the detector for feature extraction to obtain low-level features and high-level features of the training data set; The low-level features and high-level features are fused based on texture suppression to obtain fused features; Input the fused features into the FPN network to construct a feature pyramid; Input the features of each layer of the feature pyramid into the classification subnet and regression subnet respectively to detect defect targets of different sizes and obtain training detection information; Compare the training detection information with the corresponding defect labels and adjust the detector model parameters; The step of fusing low-level features and high-level features based on texture suppression to obtain fused features specifically includes: The five residual blocks of the feature extraction network ResNet are used as the input of the feature fusion module; Performing channel conversion on the output of the first residual block and the output of the second residual block based on convolution to obtain the first residual block and the second residual block having the same number of channels as the low-level features output by the fourth residual block and the fifth residual block; Downsampling the first residual block output and the second residual block output respectively based on the ReLU activation function to obtain the downsampled first residual block output and the second residual block output; Compare the downsampled first residual block output with the fourth residual block output feature point by feature point and take the intersection to obtain a first intersection feature; Compare the downsampled second residual block output with the fifth residual block output on a feature point-by-feature basis and take the intersection to obtain a second intersection feature; The output of the third residual block is used as the first feature, the first intersection feature and the output of the fourth residual block are added as the second feature, and the second intersection feature and the output of the fifth residual block are added as the third feature, which is input into the subsequent feature pyramid network.

7. A defect detection device for furniture panels, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the defect detection method for furniture panels as described in any one of claims 1 to 5.

Images