Method, device, equipment and storage medium for detecting surface defects of particleboard

Abstract

The embodiment of the present invention provides a method, device, equipment and storage medium for particleboard surface defect detection. The method includes: determining the sample feature map to be detected corresponding to each sample image based on a feature extraction network and a feature fusion network; determining the prediction frame and prediction vector corresponding to each prior frame in the sample feature map to be detected based on a defect detection model; extracting the prototype mask corresponding to the reference sample feature map through upsampling and multi-layer convolution operations; and determining the mask matrix corresponding to the target to be detected in each prediction frame in combination with the prediction vector corresponding to the reference prediction frame; and determining the segmentation result of the sample image based on the mask matrix. The defect detection model provided by the present invention can identify and frame defects, and accurately determine the boundaries of defects by using the linear combination of the mask coefficient of the prediction frame and the prototype mask of the sample feature map to be detected with the maximum partitioning scale, thereby improving the accuracy of particleboard defect identification and framing and the accuracy of segmentation.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, equipment and storage medium for detecting surface defects of particleboard. Background Art

[0002] Particleboard is widely used in furniture making, decoration and renovation, floor base material production, composite door manufacturing and other industries. Particleboard is an artificial board made of wood or other cellulose materials made of scraps glued together. During the production process of particleboard continuous press, due to the influence of various factors such as raw material quality and production process, some product boards may have defects such as glue spots, glue blocks, shavings, dust spots, oil stains, and core leakage. Surface defects will reduce the strength of the board, affect secondary processing, and bring economic losses to the company.

[0003] Traditional inspection methods rely on manual visual inspection for defects, but due to long hours of work, quality inspection workers are prone to visual fatigue, resulting in high rates of missed detection and false detection. Furthermore, the speed of manual inspection is lower than the speed of the production line, and the production line can only be inspected at low speeds with manual inspection, which greatly limits the production efficiency of particleboard. Existing inspection methods mostly use image processing technology and defect feature extraction to identify defects and then classify them. Defect detection is a two-step process, which requires a large amount of computation and cannot accurately calculate the target area of defects on the particleboard surface, hindering the refined management of factory production.

[0004] Therefore, improving the accuracy of particleboard surface defect identification and framing as well as the precision of segmentation has become a technical problem that needs to be urgently solved in the industry. Summary of the Invention

[0005] In response to the technical problems existing in the prior art, the present invention provides a method, device, equipment and storage medium for detecting surface defects of particleboard.

[0006] In a first aspect, the present invention provides a method for detecting surface defects of particleboard, comprising:

[0007] Based on the feature extraction network and feature fusion network in the defect detection model, a feature map of a sample to be detected corresponding to each sample image is determined; the sample image is an image of particleboard with defects; the feature map of the sample to be detected includes an image divided into grids according to three scales;

[0008] Based on the segmentation layer in the defect detection model, determine the prediction box corresponding to the prior box in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction box; the prediction vector includes a mask coefficient for representing the weight corresponding to each target to be detected in the prediction box;

[0009] Extracting a prototype mask corresponding to a reference sample feature map through upsampling and multi-layer convolution operations; the reference sample feature map is the one with the largest division scale in the feature map of the sample to be detected;

[0010] Determine a mask matrix corresponding to the target to be detected within each prediction frame based on the mask coefficients in the prediction vector corresponding to the reference prediction frame and the prototype mask corresponding to the reference sample feature map; the reference prediction frame is obtained by screening the prediction frame within each grid in the feature map of the sample to be detected based on a non-maximum suppression algorithm; the mask matrix is used to represent the predicted segmentation area corresponding to the target to be detected;

[0011] Based on the mask matrix, the feature map of the sample to be detected is cropped and binarized to obtain a segmentation result of the sample image; the segmentation result is the boundary of each target to be detected in the sample image;

[0012] The defect detection model is used to anchor according to the prior frame group corresponding to the feature map of the sample to be detected at different scales, and determine the prediction frame closest to the target to be detected in the feature map of the sample to be detected, the confidence, the defect category to which the target to be detected belongs, and the boundary of the target to be detected.

[0013] Optionally, the defect detection model training method includes:

[0014] Based on the segmentation layer in the defect detection model, determine the positive and negative sample labels and prediction vectors corresponding to each prediction box; the prediction box is the prediction box corresponding to each prior box in each grid in the feature map of the sample to be detected; the positive and negative samples are used to indicate whether the target to be detected exists in the prediction box;

[0015] Determine the classification loss corresponding to the feature map of the sample to be detected based on a first predicted probability, a first actual probability, and positive and negative sample labeling parameters; the first predicted probability is the probability that the target to be detected in the predicted box belongs to the first defect category; the first defect category is any one of the defect categories; the first actual probability is the true probability that the target to be detected in the prior box corresponding to the prediction box to which the first predicted probability belongs belongs to the first defect category;

[0016] Determine the confidence loss corresponding to the feature map of the sample to be detected based on the prediction confidence in the prediction vector corresponding to the prediction box, the true confidence corresponding to the prior box corresponding to the prediction box, and the positive and negative sample labeling parameters;

[0017] Determine the segmentation loss corresponding to the feature map of the sample to be detected based on a first predicted mask matrix and a first actual mask matrix, as well as positive and negative sample labeling parameters; the first predicted mask matrix is a result of a linear combination of the mask coefficients corresponding to the predicted box and the prototype mask corresponding to the reference sample feature map; the first actual mask matrix is a real mask area corresponding to the grid to which the predicted box belongs;

[0018] Determine the bounding box regression loss corresponding to the feature map of the sample to be detected based on the position of the predicted box in the prediction vector corresponding to the predicted box and the position of the real box that overlaps with the predicted box;

[0019] Determine the bounding box regression loss corresponding to the feature map of the sample to be detected based on the intersection-over-union ratio of the predicted box and the true box corresponding to the target to be detected in the grid, the distance between the center point of the predicted box and the center point of the true box, the similarity between the aspect ratio of the predicted box and the aspect ratio of the true box, and the weight coefficient corresponding to the similarity;

[0020] The training of the defect detection model is completed when the total loss function determined by the classification loss, the confidence loss, the segmentation loss and the bounding box regression loss is minimized.

[0021] Optionally, determining the feature map of the sample to be detected corresponding to each sample image based on the feature extraction network and the feature fusion network in the defect detection model includes:

[0022] The feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample image to obtain first feature maps of three different division scales;

[0023] Based on a deep convolutional network, feature aggregation is performed on the concatenation of the upsampled second feature map and the third feature map to determine a first aggregation result; the second feature map is the result of convolution processing on the smallest-scale segmentation in the first feature map; the third feature map is the second-smallest-scale segmentation in the first feature map;

[0024] Based on a shallow convolutional network, feature aggregation is performed on the concatenation of the upsampled first aggregation result and the fourth feature map, a second aggregation result is determined, and the feature map of the sample to be detected with the largest division scale is output to the segmentation layer; the fourth feature map is the one with the largest division scale in the first feature map;

[0025] Performing convolution processing on the concatenation result of the second aggregation result and the first aggregation result to determine a third aggregation result, and outputting the third aggregation result as the feature map of the sample to be detected with the second largest division scale to the segmentation layer;

[0026] A convolution process is performed on the splicing result of the third aggregation result and the second feature map, and the feature map of the sample to be detected with the smallest division scale is output to the segmentation layer.

[0027] Optionally, the feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample feature map to obtain three first feature maps with different division scales, including:

[0028] Based on the multi-layer convolutional network in the feature extraction network, downsampling the labeled sample image to obtain the sample images at different sampling rates;

[0029] Based on the coordinate attention mechanism, the correlation relationship between the position information and channels of the sample image at different sampling rates is established to obtain the first feature maps of three different division scales;

[0030] Based on the simplified spatial pyramid pooling layer, the first feature maps of the three different division scales are converted into feature maps of a fixed scale, and the first feature map with the smallest division scale is updated.

[0031] Optionally, before the feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample feature map to obtain the first feature maps of three different division scales, the method includes:

[0032] Performing a preprocessing operation on the sample image; the preprocessing operation includes: uniform pixel cropping and sample expansion;

[0033] The defects in the pre-processed sample image are annotated by an annotation tool; the annotation includes annotating the defect category, the real mask area corresponding to the defect, and the real confidence level corresponding to the defect.

[0034] Optionally, the prediction vector includes prediction confidence, the position of the prediction box, the predicted defect category, and a mask coefficient, wherein the position of the prediction box is represented by the horizontal coordinate, vertical coordinate, width, and height of the center point of the prediction box; the predicted defect category is used to represent the characteristic value of the target to be detected in the prediction box belonging to each defect category.

[0035] Optionally, determining the bounding box regression loss corresponding to the feature map of the sample to be detected based on the position of the predicted box in the prediction vector corresponding to the predicted box and the position of the real box overlapping with the predicted box includes:

[0036] Based on the position of the predicted frame and the position of the real frame, respectively determining the intersection-over-union ratio of the predicted frame and the real frame, the distance between the center point of the predicted frame and the center point of the real frame, and the similarity between the aspect ratio of the predicted frame and the aspect ratio of the real frame;

[0037] Determining a weight coefficient corresponding to the similarity based on the similarity and the intersection-over-union ratio;

[0038] Determine the diagonal distance of the minimum rectangular box including the predicted box and the true box as the diagonal distance;

[0039] Determining a regression loss corresponding to a single prediction box based on the intersection-over-union ratio, the distance between the center points, the diagonal distance, the similarity, and a weight coefficient corresponding to the similarity;

[0040] Based on all corresponding prediction boxes in the feature map of the sample to be detected and the regression loss corresponding to a single prediction box, the regression loss of the bounding box corresponding to the feature map of the sample to be detected is determined.

[0041] Optionally, the first prediction probability is a result of processing a characteristic value of the predicted defect category in the prediction vector corresponding to the prediction frame using a normalized exponential function.

[0042] Optionally, the classification loss, the confidence loss and the segmentation loss are determined based on a binary cross entropy loss function.

[0043] In a second aspect, the present invention further provides a device for detecting surface defects of particleboard, comprising:

[0044] A feature map module is configured to determine a feature map of a sample to be detected corresponding to each sample image based on a feature extraction network and a feature fusion network in a defect detection model; the sample image is an image of particleboard with defects; and the feature map of the sample to be detected includes an image divided into grids according to three scales;

[0045] A prediction frame module is used to determine, based on the segmentation layer in the defect detection model, the prediction frame corresponding to the prior frame in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction frame; the prediction vector includes a mask coefficient used to represent the weight corresponding to each target to be detected in the prediction frame;

[0046] An extraction module is used to extract a prototype mask corresponding to a reference sample feature map through upsampling and multi-layer convolution operations; the reference sample feature map is the one with the largest division scale in the feature map of the sample to be detected;

[0047] A determination module, configured to determine a mask matrix corresponding to the target to be detected within each prediction frame based on the mask coefficients in the prediction vector corresponding to the reference prediction frame and the prototype mask corresponding to the reference sample feature map; the reference prediction frame is obtained by screening the prediction frame within each grid in the feature map of the sample to be detected based on a non-maximum suppression algorithm; the mask matrix is used to represent the predicted segmentation area corresponding to the target to be detected;

[0048] A segmentation module, configured to perform cropping and binarization processing on the feature map of the sample to be detected based on the mask matrix to obtain a segmentation result of the sample image; the segmentation result is the boundary of each target to be detected in the sample image;

[0049] The defect detection model is used to anchor according to the prior frame group corresponding to the feature map of the sample to be detected at different scales, and determine the prediction frame closest to the target to be detected in the feature map of the sample to be detected, the confidence, the defect category to which the target to be detected belongs, and the boundary of the target to be detected.

[0050] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting surface defects of particleboard as described in the first aspect above is implemented.

[0051] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting surface defects of particleboard as described in the first aspect above.

[0052] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for detecting surface defects of particleboard as described in the first aspect above.

[0053] The method, device, equipment and storage medium for detecting surface defects of particleboard provided by the present invention determine the multi-scale feature map of the sample to be detected corresponding to the particleboard with defects, and use the defect detection model to anchor the feature map of the sample to be detected with the corresponding prior frame group, determine the prediction frame corresponding to each prior frame, and the prediction vector of the prediction frame, use non-maximum suppression to filter the prediction frame, and based on the prediction vector corresponding to the filtered prediction frame, combine the prototype mask determined by the feature map of the sample to be detected with the largest division scale, determine the mask matrix corresponding to each target to be detected, and accurately segment the defects in the particleboard through the mask matrix. The present invention obtains the information-rich multi-scale feature map of the sample to be detected by using the segmentation layer of the defect detection model, determines the prediction vector corresponding to each prediction frame, and determines the mask matrix based on the mask coefficient in the prediction vector and the prototype mask, which is used to accurately classify and segment the defects in the particleboard, with high computational efficiency and more accurate defect identification, framing and segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 1 is a flow chart of a method for detecting surface defects of particleboard provided by an embodiment of the present invention;

[0056] Figure 2 Schematic diagram of a particleboard surface image data acquisition device provided by an embodiment of the present invention;

[0057] Figure 3 Schematic diagram of the structure of the feature fusion network in the defect detection model provided by an embodiment of the present invention;

[0058] Figure 4 Schematic diagram of the structure of the Conv module in the feature extraction network provided by an embodiment of the present invention;

[0059] Figure 5 Schematic diagram of the structure of the Coord Att module in the feature extraction network provided by an embodiment of the present invention;

[0060] Figure 6 Schematic diagram of the structure of the SimSPPF module in the feature extraction network provided by an embodiment of the present invention;

[0061] Figure 7is a schematic diagram of the training phase of the defect detection model provided by an embodiment of the present invention;

[0062] Figure 8 This is a schematic diagram of the implementation process of particleboard surface defect detection provided by an embodiment of the present invention;

[0063] Figure 9 This is one of the schematic diagrams comparing simulation results of the defect detection model and the YOLOv5s model provided by an embodiment of the present invention;

[0064] Figure 10 This is the second schematic diagram of the simulation comparison results of the defect detection model provided by an embodiment of the present invention and the YOLOv5s model;

[0065] Figure 11 This is the third schematic diagram of the simulation comparison results of the defect detection model provided by an embodiment of the present invention and the YOLOv5s model;

[0066] Figure 12 This is the fourth schematic diagram of the simulation comparison results of the defect detection model provided by an embodiment of the present invention and the YOLOv5s model;

[0067] Figure 13 This is the fifth schematic diagram of the simulation comparison results of the defect detection model provided by an embodiment of the present invention and the YOLOv5s model;

[0068] Figure 14 is a schematic diagram of the particleboard surface defect detection results provided by an embodiment of the present invention;

[0069] Figure 15 1 is a schematic structural diagram of a device for detecting surface defects of particleboard provided by an embodiment of the present invention;

[0070] Figure 16 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In embodiments of the present invention, the term "and / or" describes the association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0072] In the embodiments of the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0073] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention and not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0074] Figure 1 FIG. 1 is a flow chart of a method for detecting surface defects of particleboard provided by an embodiment of the present invention, as shown in FIG. Figure 1 As shown, the method includes:

[0075] Step 101: Based on the feature extraction network and feature fusion network in the defect detection model, determine the sample feature map to be detected corresponding to each sample image; the sample image is an image of particleboard with defects; the sample feature map to be detected includes an image divided into grids according to three scales; the defect detection model is used to anchor according to the prior frame group corresponding to the sample feature map to be detected at different scales, and determine the prediction box closest to the target to be detected in the sample feature map to be detected, the confidence, the defect category to which the target to be detected belongs, and the boundary of the target to be detected.

[0076] Specifically, to obtain images of defective particleboards, multiple high-resolution industrial line array cameras arranged on the particleboard production line can be used to capture images of the upper and lower surfaces of the particleboards in real time. The resolution of the industrial camera can be 4096 pixels, 8192 pixels or higher. Figure 2 FIG. 1 is a schematic diagram of a particleboard surface image data acquisition device provided by an embodiment of the present invention. Figure 2 As shown, when the production line's conveyor belt delivers particleboard to the corresponding imaging area, it triggers the light source above the particleboard surface to illuminate. For example, photoelectric sensors are placed at both ends of the imaging position. When the particleboard enters the edge of the imaging area, the photoelectric sensors trigger a pulse signal, which controls the LED light strip to turn on and the industrial line scan camera to capture images in a uniformly illuminated environment. By stitching the images from the two cameras, image data of the complete particleboard surface can be obtained. Through simple image recognition or manual screening, images of particleboards with defects are selected as sample images.

[0077] The sample image is input into the defect detection model. The feature extraction network in the defect detection model extracts features from the sample image to obtain as much image feature information as possible to characterize the sample image. For example, the original sample image is sampled at different sampling rates, and image feature information from different channels is swapped without changing the size of the feature map, outputting feature maps of different scales. Furthermore, the feature fusion network in the defect detection model establishes associations between these feature maps of different scales. For example, these feature maps of different scales are downsampled and feature concatenated using a convolutional network configured with different tensor information. Thus, a sample feature map to be detected corresponding to each sample image is obtained. The sample feature map to be detected can include grid images of multiple scales. The specific number of scales can be determined based on the specific hierarchical structure of the feature extraction and feature fusion networks. Images divided into grids of three scales are commonly used. The present invention is primarily illustrated using a sample feature map to be detected composed of images divided into grids of three scales as an example. For images divided into grids of more scales, the specific implementation method can be similar to that for images divided into grids of three scales and will not be further described here.

[0078] Step 102: Based on the segmentation layer in the defect detection model, determine the prediction box corresponding to the prior box in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction box; the prediction vector includes a mask coefficient for representing the weight corresponding to each target to be detected in the prediction box;

[0079] After obtaining the feature map of the sample to be detected corresponding to each sample image through the above steps, the segmentation layer of the defect detection model provided by the present invention is input. In this segmentation layer, there are corresponding prior frame groups for grid images divided at different scales. The prior frame group is composed of multiple prior frames of different sizes, and the size of each prior frame is smaller than the size of the grid. For example, for the feature map of the sample to be detected with a grid size of 160×160, the corresponding prior frame group can be composed of three prior frames of 10×13, 16×30 and 33×23, or four prior frames of 10×13, 16×27, 21×13 and 33×23, or five prior frames of different sizes; for the feature map of the sample to be detected with a grid size of 40×40, the corresponding prior frame group can be composed of three prior frames of 116×90, 156×198 and 373×326, or four prior frames of 116×90, 156×198, 232×187 and 373×326. Of course, in the segmentation layer of the defect detection model of the present invention, the number of prior frames included in the prior frame group used for the feature maps of the samples to be detected with different grid sizes is consistent. For example, the grid sizes are divided into three types: A, B, and C. The number of prior frames in the prior frame group corresponding to the feature map of the sample to be detected with grid size A is 5, so the number of prior frames in the prior frame group corresponding to the feature map of the sample to be detected with grid sizes B and C is also 5.

[0080] After the defect detection model is trained, it can make predictions based on the feature map of the target sample, which is composed of an input image divided into a grid at three scales. The prediction box corresponding to each prior box is determined, as well as the corresponding prediction vector. This prediction vector represents the confidence of the prediction box, the position of the prediction box, the defect category corresponding to the target in the prediction box, and the mask coefficient corresponding to the prediction box. The confidence of the prediction box can be understood as the degree of confidence or score of the presence of the target within the prediction box. The position of the prediction box can be understood as the size of the prediction box, specifically its location in the feature map of the target sample, such as coordinates, width, and height. The defect category corresponding to the target in the prediction box can be understood as the value of the probability that the target in the prediction box will be predicted as each defect category. For example, if there are five defect categories, the defect category corresponding to the target in the prediction box is represented by a five-digit number, such as {2, 4, 5, 1, 3}. Each digit represents a defect category, and the value of each digit indicates the probability that the target belongs to the defect category corresponding to that digit. A larger value indicates a greater probability. The mask coefficient corresponding to the prediction frame can be understood as distinguishing between various image features within the prediction frame, such as those representing the background, the human body, and trees. Different weights are assigned based on the features to be extracted. For example, if the human body is to be extracted, a higher value is assigned to the mask coefficient representing the human body. In this invention, for defects on the particleboard surface, different weights can be assigned based on the degree of contamination, such as dust belonging to the surface layer and glue spots belonging to the deeper layers, thereby enabling detection of different defect categories.

[0081] Step 103: extracting a prototype mask corresponding to a reference sample feature map through upsampling and multi-layer convolution operations; the reference sample feature map is the one with the largest division scale in the feature map of the sample to be detected;

[0082] After obtaining the prediction box corresponding to the prior box in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction box, it is also necessary to obtain the corresponding prototype mask. The prototype mask can be understood as dividing the image into different layers. Each layer has corresponding mask information to match it, and the prototype mask includes the mask information corresponding to all layers.

[0083] When obtaining the prototype mask, the largest scale in the feature map of the sample to be detected is selected as the reference sample feature map. Upsampling and multi-layer convolution operations are performed on the reference sample feature map. This can retain more feature information of the original image and reduce the loss of image feature information. The resulting prototype mask contains richer information and is closer to the actual sample image. In addition, upsampling can also capture as much feature information as possible from the original image.

[0084] Step 104: Determine a mask matrix corresponding to the target to be detected within each prediction frame based on the mask coefficients in the prediction vector corresponding to the reference prediction frame and the prototype mask corresponding to the reference sample feature map; the reference prediction frame is obtained by screening the prediction frame within each grid in the target sample feature map based on a non-maximum suppression algorithm; the mask matrix is used to represent the predicted segmentation region corresponding to the target to be detected;

[0085] After determining the prediction box corresponding to each prior box and the prediction vector corresponding to the prediction box, for a single grid, there may be multiple prediction vectors corresponding to the prediction box. In order to select the prediction box closest to the target to be detected, the non-maximum suppression algorithm is used to filter out the prediction box closest to the target to be detected from the multiple prediction boxes corresponding to each grid. This is used as the reference prediction box. Based on the prediction vector corresponding to the reference prediction box and the prototype mask, the mask matrix corresponding to the target to be detected within the prediction box is determined. For example, by linearly combining the mask coefficients in the prediction vector and the prototype mask, the mask matrix corresponding to the target to be detected within the prediction box can be obtained, which is the instance mask result.

[0086] Step 105 : Based on the mask matrix, the feature map of the sample to be detected is cropped and binarized to obtain a segmentation result of the sample image; the segmentation result is the boundary of each target to be detected in the sample image.

[0087] Based on the mask matrix obtained in the above steps, the feature map of the sample to be detected is cropped. This can be understood as clearing the pixels in the mask matrix that are zero, according to the pixels corresponding to each element in the mask matrix. The subsequent binarization process can be understood as binarizing the generated instance mask with a threshold of 0.5 after the cleanup, thus obtaining the segmentation result corresponding to the sample image.

[0088] The method for detecting surface defects of particleboard provided by the present invention determines a multi-scale feature map of a sample to be detected corresponding to a particleboard with defects, and anchors the feature map of the sample to be detected with a corresponding prior frame group using a defect detection model, determines a prediction frame corresponding to each prior frame, and a prediction vector of the prediction frame, uses non-maximum suppression to filter the prediction frame, and based on the prediction vector corresponding to the filtered prediction frame, combines the prototype mask determined by the feature map of the sample to be detected with the largest division scale, determines a mask matrix corresponding to each target to be detected, and accurately segments the defects in the particleboard through the mask matrix. The present invention obtains an information-rich multi-scale feature map of the sample to be detected by using a feature extraction network and a feature fusion network, uses a defect detection model to determine the prediction vector corresponding to each prediction frame, and determines a mask matrix based on the mask coefficient in the prediction vector and the prototype mask, for accurately classifying and segmenting the defects in the particleboard, with high computational efficiency and more accurate defect identification, framing and segmentation.

[0089] Optionally, the prediction vector includes prediction confidence, the position of the prediction box, the predicted defect category, and a mask coefficient, wherein the position of the prediction box is represented by the horizontal coordinate, vertical coordinate, width, and height of the center point of the prediction box; the predicted defect category is used to represent the characteristic value of the target to be detected in the prediction box belonging to each defect category.

[0090] Specifically, the prediction vector includes prediction confidence, the position of the prediction box, the predicted defect category, and the mask coefficient.

[0091] The prediction confidence indicates the confidence corresponding to the prediction box. The position of the prediction box can be expressed as the horizontal and vertical coordinates of the center point of the prediction box, the width and height of the prediction box, or the horizontal and vertical coordinates of the center point and the horizontal and vertical coordinates of the four vertices of the box. It is used to indicate the specific position of this prediction box in the feature map of the sample to be detected. The predicted defect category can occupy multiple element bits in the prediction vector. Each element bit indicates that the characteristic value of the target to be detected in the prediction box belongs to a defect type. The element bit with the largest characteristic value indicates that the probability that the target to be detected belongs to the defect type indicated by the element bit is the highest. The mask coefficient indicates the weight coefficient corresponding to the target to be detected in the prediction box, which is used to distinguish different targets to be detected in the same feature map, which is conducive to the subsequent extraction of the target object to be extracted.

[0092] The prediction vector can be expressed as:

[0093]

[0094] Among them, C i,j represents the prediction confidence of the j-th prediction box of the i-th grid, Represent the horizontal coordinate of the center point of the j-th prediction box of the i-th grid, the vertical coordinate of the center point, the width of the box, and the height of the box respectively; The feature value indicating whether the j-th prediction box of the i-th grid belongs to any of the n defect categories; Represents the mask coefficient corresponding to the jth prediction box of the i-th grid. The mask coefficient can be represented by multiple elements, that is, the value of k can be set according to the application scenario and actual needs. According to actual verification results, when the value of k is 32, the corresponding prediction effect is the best. Therefore, the mask coefficient in this invention is 32 bits.

[0095] Optionally, the defect detection model training method includes:

[0096] Based on the segmentation layer in the defect detection model, determine the positive and negative sample labels and prediction vectors corresponding to each prediction box; the prediction box is the prediction box corresponding to each prior box in each grid in the feature map of the sample to be detected; the positive and negative samples are used to indicate whether the target to be detected exists in the prediction box;

[0097] Determine the classification loss corresponding to the feature map of the sample to be detected based on a first predicted probability, a first actual probability, and positive and negative sample labeling parameters; the first predicted probability is the probability that the target to be detected in the predicted box belongs to the first defect category; the first defect category is any one of the defect categories; the first actual probability is the true probability that the target to be detected in the prior box corresponding to the prediction box to which the first predicted probability belongs belongs to the first defect category;

[0098] Determine the confidence loss corresponding to the feature map of the sample to be detected based on the prediction confidence in the prediction vector corresponding to the prediction box, the true confidence corresponding to the prior box corresponding to the prediction box, and the positive and negative sample labeling parameters;

[0099] Determine the segmentation loss corresponding to the feature map of the sample to be detected based on a first predicted mask matrix and a first actual mask matrix, as well as positive and negative sample labeling parameters; the first predicted mask matrix is a result of a linear combination of the mask coefficients corresponding to the predicted box and the prototype mask corresponding to the reference sample feature map; the first actual mask matrix is a real mask area corresponding to the grid to which the predicted box belongs;

[0100] Determine the bounding box regression loss corresponding to the feature map of the sample to be detected based on the position of the predicted box in the prediction vector corresponding to the predicted box and the position of the real box that overlaps with the predicted box;

[0101] Determine the bounding box regression loss corresponding to the feature map of the sample to be detected based on the intersection-over-union ratio of the predicted box and the true box corresponding to the target to be detected in the grid, the distance between the center point of the predicted box and the center point of the true box, the similarity between the aspect ratio of the predicted box and the aspect ratio of the true box, and the weight coefficient corresponding to the similarity;

[0102] The training of the defect detection model is completed when the total loss function determined by the classification loss, the confidence loss, the segmentation loss and the bounding box regression loss is minimized.

[0103] Specifically, the above-mentioned defect detection model is used to detect surface defects of particleboard, which is implemented after the defect detection model is trained. The training method of the defect detection model specifically includes:

[0104] Through the segmentation layer in the defect detection model, the feature map of the sample to be detected corresponding to the sample image is predicted. That is, the image is divided into grids of different scales, and the corresponding prediction frame group is used to anchor the target to be detected. If there is a prediction frame of the target to be detected, it is marked as a positive sample. If there is no prediction frame of the target to be detected, it is marked as a negative sample. At the same time, the prediction vector corresponding to each prediction frame can also be obtained.

[0105] The total loss function of the defect detection model is further determined, and the training of the defect detection model is completed when the total loss function is minimized. The total loss function includes classification loss, confidence loss, segmentation loss, and bounding box regression loss.

[0106] The classification loss, the confidence loss, and the segmentation loss are determined based on a binary cross entropy loss function.

[0107] The method for determining the classification loss specifically includes: according to the defect categories included in each prediction vector, the probabilities of the various defect categories corresponding to the prediction box to which the prediction vector belongs can be determined as the first prediction probability. For example, the probability of the prediction box corresponding to five defect categories is {ρ1, ρ2, ρ3, ρ4, ρ5}, and ρ1<ρ2<ρ5<ρ3<ρ4, then the probability values of the five defect categories corresponding to the first prediction probability are determined to be ρ1, ρ2, ρ3, ρ4 and ρ5 respectively. The probability of the defect category corresponding to the target to be detected in the prediction box has been marked, that is, the first actual probability. Then, based on the first prediction probability and the first actual probability, as well as the corresponding positive and negative sample labels, the classification loss L corresponding to the feature map of the sample to be detected is determined. class In the process of training the defect detection model, the classification loss is reduced, that is, the classification of the predicted particleboard surface defects is made more consistent with the actual defect classification, thereby improving the accuracy of the defect detection model in classifying particleboard surface defects.

[0108] Optionally, the first prediction probability is a result of processing a characteristic value of the predicted defect category in the prediction vector corresponding to the prediction frame using a normalized exponential function.

[0109] Specifically, the feature value of the predicted defect category in the prediction vector is expressed as The corresponding first prediction probability is determined by using a normalized exponential function to process the characteristic value of the predicted defect category in the prediction vector, which can be expressed as follows:

[0110]

[0111] Where, ρ i,j (r) represents the predicted probability value of the j-th prediction box of the i-th grid belonging to the defect category r, softmax() represents the normalized exponential function, n represents the total number of defect categories, Indicates the eigenvalue corresponding to the case where the object to be detected in the j-th prediction box of the i-th grid belongs to the l-th defect category.

[0112] The method for determining the confidence loss includes: determining the confidence loss corresponding to the feature map of the sample to be detected based on the prediction confidence in the prediction vector corresponding to the prediction box, the true confidence corresponding to the prior box corresponding to the prediction box, and the positive and negative sample labeling parameters; for example, within all grids of the feature map of the sample to be detected, determining the gap between the confidence of the prediction box of the positive sample and its corresponding true confidence, and the gap between the confidence of the prediction box of the negative sample and its corresponding true confidence, performing weighted summation, and using this as the confidence loss of the defect detection model. By reducing the confidence loss, the positive samples in the sample image are made to approach the defective area in the original image, and the negative samples are reduced to move away from the defective area in the original image.

[0113] The method for determining the segmentation loss includes: determining the segmentation loss corresponding to the feature map of the sample to be detected based on the first predicted mask matrix and the first actual mask matrix, as well as the positive and negative sample labeling parameters; the first predicted mask matrix is the result of a linear combination of the mask coefficient corresponding to the predicted box and the prototype mask corresponding to the reference sample feature map; the first actual mask matrix is the real mask area corresponding to the grid to which the predicted box belongs; it can be understood that in all grids of the feature map of the sample to be detected, the linear combination result of the mask coefficient corresponding to each predicted box in the positive sample and the prototype mask corresponding to the reference sample feature map is determined as the first predicted mask matrix, the first actual mask matrix is the result of a linear combination of the mask coefficient corresponding to the predicted box in the positive sample and the prototype mask corresponding to the reference sample feature map. A prediction mask matrix can be used to divide the boundaries of the target to be detected within the prediction box, so that each positive sample corresponds to a first prediction mask matrix, and there is a corresponding first actual mask matrix corresponding to the positive sample. The first actual mask matrix is a mathematical representation of the actual boundary of the defect to be detected corresponding to the positive sample. By determining the gap between the first prediction mask matrix and the first time mask matrix corresponding to each positive sample, and minimizing the gap between the two as much as possible, the predicted result is closer to the boundary of the real defect, thereby improving the accuracy of the above-mentioned defect detection model for predicting the boundaries of various defects on the particleboard surface, that is, improving the accuracy of the particleboard surface defect segmentation.

[0114] The method for determining the bounding box regression loss includes: determining the bounding box regression loss corresponding to the feature map of the sample to be detected based on the position of the prediction box in the prediction vector corresponding to the prediction box and the position of the real box that overlaps with the prediction box. Here, the position of the prediction box and the position of the real box both include information such as the coordinates of the four vertices, the coordinates of the center point, the width and height. Through the positional relationship between the prediction box and the real box corresponding to the prediction box, the relative distance between the two boxes, the overlapping area between the two boxes, the total area enclosed by the two boxes, the similarity between the two boxes and other parameters can be determined, and then these parameters are combined to determine the bounding box regression loss. During the training process, the bounding box regression loss is minimized to make the prediction box closer to the real box, thereby improving the accuracy of the particleboard surface defect framing.

[0115] By determining the minimum of the total loss function, that is, making the classification loss, the confidence loss, the segmentation loss and the bounding box regression loss as minimum, a trained defect detection model is obtained. Of course, in the actual training process, after a certain number of trainings, the trend of the total loss function of the defect detection model becoming smaller does not change significantly, that is, the value of the total loss function of the defect detection model reaches convergence, and it can also be determined that the training of the defect detection model is complete. It can be understood that the standard ideal state for the completion of defect detection model training is to minimize the total loss function of the defect detection model. In actual applications, it can also be judged by judging whether the number of trainings meets the preset training number threshold and determining whether the total loss function of the defect detection model converges. When the preset training number threshold is met or the total loss function meets convergence, it can be determined that the defect detection model training is complete.

[0116] Optionally, determining the bounding box regression loss corresponding to the feature map of the sample to be detected based on the position of the predicted box in the prediction vector corresponding to the predicted box and the position of the real box overlapping with the predicted box includes:

[0117] Based on the position of the predicted frame and the position of the real frame, respectively determining the intersection-over-union ratio of the predicted frame and the real frame, the distance between the center point of the predicted frame and the center point of the real frame, and the similarity between the aspect ratio of the predicted frame and the aspect ratio of the real frame;

[0118] Determining a weight coefficient corresponding to the similarity based on the similarity and the intersection-over-union ratio;

[0119] Determine the diagonal distance of the minimum rectangular box including the predicted box and the true box as the diagonal distance;

[0120] Determining a regression loss corresponding to a single prediction box based on the intersection-over-union ratio, the distance between the center points, the diagonal distance, the similarity, and a weight coefficient corresponding to the similarity;

[0121] Based on all corresponding prediction boxes in the feature map of the sample to be detected and the regression loss corresponding to a single prediction box, the regression loss of the bounding box corresponding to the feature map of the sample to be detected is determined.

[0122] Specifically, when determining the bounding box regression loss of the defect detection model, it is mainly based on the position of the predicted box in the prediction vector corresponding to the predicted box, and the position of the real box that overlaps with the predicted box. The specific implementation can determine the relative distance between the two boxes, the overlapping area between the two boxes, the total area enclosed by the two boxes, the similarity between the two boxes, and other parameters through the positional relationship between the predicted box and the real box corresponding to the predicted box. Then, these parameters are combined to determine the bounding box regression loss by weighted summation, or the bounding box regression loss is determined by performing addition and subtraction operations on these parameters. The present invention obtains a better method for determining the bounding box regression loss through simulation, including:

[0123] First determine the single prediction box regression loss CIOU loss function:

[0124] Based on the position of the predicted frame and the position of the real frame, the intersection-over-union ratio of the predicted frame and the real frame, the distance between the center point of the predicted frame and the center point of the real frame, and the similarity between the aspect ratio of the predicted frame and the aspect ratio of the real frame are determined respectively. The formula corresponding to the intersection-over-union ratio is expressed as:

[0125]

[0126] Where, IoU i,j represents the intersection-over-union ratio of the j-th prediction box of the i-th grid in the feature map of the sample to be detected and the real box corresponding to the j-th prediction box of the i-th grid, b i,j represents the j-th prediction box of the i-th grid, represents the ground-truth box corresponding to the j-th predicted box of the i-th grid.

[0127] The formula corresponding to the similarity is expressed as:

[0128]

[0129] Where, ν i,j Represents the similarity between the aspect ratio of the j-th predicted box of the i-th grid in the feature map of the sample to be detected and the aspect ratio of the real box corresponding to the j-th predicted box of the i-th grid; Represent the width and height of the j-th prediction box of the i-th grid respectively; Respectively represent the width and height of the real box corresponding to the j-th predicted box of the i-th grid; They represent the aspect ratio of the real box and the aspect ratio of the predicted box respectively.

[0130] The distance between the center point of the predicted box and the center point of the real box can be expressed as:

[0131]

[0132] Where, Represents the distance between the center point of the jth predicted box of the i-th grid in the feature map of the sample to be detected and the center point of the real box corresponding to the jth predicted box of the i-th grid. i,j represents the j-th prediction box of the i-th grid, represents the true box corresponding to the j-th predicted box of the i-th grid; Represent the horizontal and vertical coordinates of the center point of the j-th prediction box of the i-th grid respectively; They represent the horizontal and vertical coordinates of the center point of the real box corresponding to the j-th predicted box of the i-th grid.

[0133] Based on the similarity and the intersection-over-union ratio, a weight coefficient corresponding to the similarity is determined; the weight coefficient can be expressed as:

[0134]

[0135] Where, α i,j Represents similarity ν i,j The corresponding weight coefficient.

[0136] Based on the intersection-over-union ratio, the distance between the center points, the diagonal distance, the similarity, and the weight coefficient corresponding to the similarity, the regression loss corresponding to the single prediction box is determined; it can be expressed as:

[0137]

[0138] Where, represents the regression loss corresponding to the j-th prediction box in the i-th grid of the sample feature map to be detected, c i,j Represents the diagonal distance between the minimum rectangular box including the j-th predicted box of the i-th grid and the ground-truth box corresponding to the j-th predicted box of the i-th grid.

[0139] Based on all the corresponding prediction boxes in the feature map of the sample to be detected and the regression loss corresponding to a single prediction box, the regression loss of the bounding box corresponding to the feature map of the sample to be detected is determined. It can be expressed as:

[0140]

[0141] Where, L CIoU Represents the bounding box regression loss corresponding to the feature map of the sample to be detected, Indicates whether the j-th prediction box of the i-th grid in the feature map of the sample to be detected contains the target to be detected, and B represents the total number of prediction boxes included in the i-th grid; s 2It means that the feature map of the sample to be detected is divided into s*s grids at the segmentation layer, that is, the specific scale of the grid.

[0142] Optionally, determining the feature map of the sample to be detected corresponding to each sample image based on the feature extraction network and the feature fusion network in the defect detection model includes:

[0143] The feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample image to obtain first feature maps of three different division scales;

[0144] Based on a deep convolutional network, feature aggregation is performed on the concatenation of the upsampled second feature map and the third feature map to determine a first aggregation result; the second feature map is the result of convolution processing on the smallest-scale segmentation in the first feature map; the third feature map is the second-smallest-scale segmentation in the first feature map;

[0145] Based on a shallow convolutional network, perform feature aggregation on the concatenation of the upsampled first aggregation result and the fourth feature map, determine a second aggregation result, and output the feature map of the sample to be detected with the largest division scale; the fourth feature map is the one with the largest division scale in the first feature map;

[0146] Performing convolution processing on the concatenation result of the second aggregation result and the first aggregation result to determine a third aggregation result, and outputting the third aggregation result as the feature map of the sample to be detected with the second largest division scale to the segmentation layer;

[0147] A convolution process is performed on the splicing result of the third aggregation result and the second feature map, and the feature map of the sample to be detected with the smallest division scale is output to the segmentation layer.

[0148] Specifically, the feature extraction network and feature fusion network in the defect detection model provided by the present invention are both constructed using an improved YOLOv5s algorithm. Using this feature extraction network, the sample image is processed through downsampling, feature concatenation, and feature fusion to obtain first feature maps at three different division scales. The different division scales here refer to dividing the sample image into different grids. For example, if the original image is 1280*1280 pixels, the grids used for division include 160×160, 80×80, and 40×40, or 80×80, 40×40, and 20×20, or 320×320, 160×160, and 80×80, etc. The specific division grid can be dynamically set based on the pixel size of the original image.

[0149] Through the feature extraction network, three first feature maps of different division scales are obtained as the input of the feature aggregation network. The feature aggregation network then performs upsampling, splicing, and multi-layer convolution on the first feature maps of the three different division scales to enhance the correlation between the feature information of the first feature maps of the three different division scales.

[0150] Figure 3 FIG. 1 is a schematic diagram of the structure of the feature fusion network in the defect detection model provided by an embodiment of the present invention. Figure 3 As shown in FIG, the feature fusion network specifically includes: a deep convolutional network, a shallow convolutional network, convolution, upsampling and splicing. The deep convolutional network and the shallow convolutional network both include a multi-layer network structure. The processing flow of the first feature map of three different input division scales based on the feature fusion network specifically includes:

[0151] The result of convolution processing on the smallest-scale segmentation in the first feature map is used as the second feature map;

[0152] Upsampling the second feature map;

[0153] The upsampled second feature map and the third feature map are concatenated, where the third feature map is the one with the second smallest division scale in the first feature map;

[0154] Perform feature aggregation on the concatenation result of the upsampled second feature map and the third feature map through a deep convolutional network to determine a first aggregation result;

[0155] Upsampling the first aggregation result;

[0156] Concatenate the first aggregation result after upsampling with the fourth feature map, where the fourth feature map is the one with the largest division scale in the first feature map;

[0157] Through the shallow convolutional network, feature aggregation is performed on the concatenation of the first aggregation result after upsampling and the fourth feature map to determine the second aggregation result. At the same time, during the processing of the shallow convolutional network, the feature map of the sample to be detected with the largest division scale is also output to the segmentation layer;

[0158] splicing the first aggregation result and the second aggregation result;

[0159] Performing convolution processing on the concatenation result of the second aggregation result and the first aggregation result to determine a third aggregation result; and outputting the third aggregation result as a feature map of the sample to be detected with the second largest division scale to the segmentation layer;

[0160] Splicing the third aggregation result and the second feature map;

[0161] The concatenation result of the third aggregation result and the second feature map is convolved to obtain a result after convolution, that is, a feature map of the sample to be detected with the smallest division scale, and the feature map of the sample to be detected with the smallest division scale is output to the segmentation layer.

[0162] The feature aggregation network in the defect detection model is constructed using an improved YOLOv5s algorithm. This simplifies the model structure, reduces the number of model parameters, and fully utilizes the feature information included in the sample images obtained by the feature extraction network. This strengthens the correlation between feature information in the first feature maps at different scales, improves the detection of smaller defects, retains more shallow position features, and further enhances the network's feature extraction capabilities for defects. This facilitates the subsequent segmentation layer to anchor, identify, and segment defects in the feature maps of the samples to be detected at the three scales.

[0163] Optionally, the feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample feature map to obtain three first feature maps with different division scales, including:

[0164] Based on the multi-layer convolutional network in the feature extraction network, downsampling the labeled sample image to obtain the sample images at different sampling rates;

[0165] Based on the coordinate attention mechanism, the correlation relationship between the position information and channels of the sample image at different sampling rates is established to obtain the first feature maps of three different division scales;

[0166] Based on the simplified spatial pyramid pooling layer, the first feature maps of the three different division scales are converted into feature maps of a fixed scale, and the first feature map with the smallest division scale is updated.

[0167] Specifically, the feature extraction network in the defect detection model is built based on the improved YOLOv5s algorithm. The corresponding network structure is relatively simple and the model has fewer parameters. This can reduce the subsequent model-based total loss function, the amount of calculation required to update the model parameters, and the time required for the adjustment process.

[0168] The feature extraction network includes a multi-layer convolutional network, which downsamples the labeled sample images at different sampling rates to obtain feature maps corresponding to the sample images at different sampling rates. It can be understood that the higher the sampling rate, the more pixels are included in the feature map at that sampling rate, and the richer the feature information. Specifically, the multi-layer convolutional network includes a Conv module and a Ghost Bottleneck module, where the structure of the Conv module is as follows: Figure 4As shown, the Conv module includes a common convolution layer Conv2d, a batch normalization operation BatchNorm2d and an activation function SiLU. The Conv module mainly implements the process of passing the input features through the convolution layer, activation function, and normalization layer to finally obtain the output layer. The Ghost Bottleneck module is a structure composed of a plurality of Ghost modules. The Ghost module is a lightweight convolutional network that can improve the detection accuracy of the network while minimizing the network calculation loss. The structure of the Ghost Bottleneck module of the present invention includes two structures with a step size of 1 and a step size of 2. The main path of the Ghost Bottleneck module structure with a step size of 1 is composed of two Ghost modules in series. The first Ghost module is used to expand the number of input feature map channels for expansion processing for subsequent operations. The second Ghost module reduces the number of output feature map channels to the same as the number of input channels, and connects the information transmission of the two Ghost modules through a diameter structure; the Ghost Bottleneck module structure with a step size of 2 has a DWConv module with a step size of 2 added between the two Ghost modules to reduce the size of the feature map to half of the input, thereby playing a downsampling role. Both structures apply batch normalization and ReLU activation function after the output of the first Ghost module, while each subsequent layer only applies batch normalization. This structural approach can effectively reduce the number of model parameters and the amount of calculation. At the same time, the Ghost module is used to optimize the feature map and improve the detection efficiency of the model.

[0169] In addition, the coordinate attention mechanism is also used to establish the correlation between the position information and channels of the sample images at each sampling rate, and obtain the first feature maps of three different division scales. For example, the Coord Att module used in the present invention introduces a new attention mechanism, namely the coordinate attention mechanism. The structure of the Coord Att module is as follows: Figure 5 As shown in the figure. The Coord Att module captures the target structure by embedding position information into channel attention. It also takes into account position and channel information, realizes the acquisition of long-range dependencies, and can fully utilize the target's position information to accurately locate the region of interest in the feature map. Its calculation process includes two parts: coordinate information embedding and coordinate attention generation. The coordinate information embedding part decomposes the global pooling into a pair of one-dimensional features for encoding operations, allowing the attention module to capture long-term dependencies along one spatial direction and preserve precise position information along another spatial direction. It can well obtain the global receptive field and encode precise position information.

[0170] Based on the above-mentioned processing steps, the first feature maps of the three different division scales are obtained, and the first feature maps of the three different division scales are converted into feature maps of fixed scale size through the simplified spatial pyramid pooling layer, and the first feature map with the smallest division scale is updated. The SimSPPF module used in the simplified spatial pyramid pooling layer is used to increase the receptive field. Unlike the original spatial pyramid pooling SPPF module, the SimSPPF module used in the present invention represents the 9*9 and 13*13 pooling kernels in the SPPF module with 5*5 pooling kernels, and at the same time changes the activation function in the Conv module from SiLU to ReLU, which can compress network parameters, reduce network calculation amount, and improve model reasoning speed. The structure of the SimSPPF module is as follows: Figure 6 shown.

[0171] The feature extraction network in the defect detection model provided by the present invention can fully extract the feature information in the sample image and establish the association between each feature information. It also introduces the coordinate attention mechanism to establish the association between the position information and the channel in the sampled image to obtain the first feature maps of three different division scales, which can fully characterize the original feature information included in the sample image, and can also reduce the amount of data for model training, thereby speeding up the module training time, and is also conducive to the trained defect detection model to obtain more accurate prediction results more quickly.

[0172] Optionally, before the feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample feature map to obtain the first feature maps of three different division scales, the method includes:

[0173] Performing a preprocessing operation on the sample image; the preprocessing operation includes: uniform pixel cropping and sample expansion;

[0174] The defects in the pre-processed sample image are annotated by an annotation tool; the annotation includes annotating the defect category, the real mask area corresponding to the defect, and the real confidence level corresponding to the defect.

[0175] Specifically, after obtaining a particleboard image with defects as a sample image, since the shapes of the same type of defects on the particleboard surface are different, and the complete board surface image data occupies a large space and pixel value, in order to ensure that the particleboard defect recognition model can correctly process the collected defective images, reduce the workload of manual annotation of sample data and the computing resource requirements during model training, the sample image needs to be preprocessed. Preprocessing operations include uniform pixel cropping and sample expansion. Among them, uniform pixel cropping mainly crops the defective areas of the sample image in sequence according to a fixed pixel size. The fixed pixel size can be 1024*2048 pixels, 2048*2048 pixels or 2048*4096 pixels. Of course, the fixed pixel size here can be dynamically adjusted according to the actual application scenario or detection requirements.

[0176] Sample expansion aims to increase the limited number of existing samples, improve the generalization ability of the defect detection model, reduce overfitting, and reduce the impact of additional factors on the recognition results. Specifically, sample data expansion can be achieved by adding Gaussian noise, vertical flipping, left-right rotation, median filtering, and random 90° rotation.

[0177] After the sample image data is preprocessed, a labeling tool is used to label the data and save the labeled labels. The labeled labels specifically include the defect type corresponding to the defect, the real frame of the defect, the boundary of the defect (real mask area) and the real confidence corresponding to the defect, etc. There may be multiple defect types. In the present invention, the collected particleboard surface defects mainly include four types: wood particles, dust, glue spots and oil stains, and leaking cores. The glue block, glue spot and oil stain defect images with similar characteristics are merged into a defect type of "glue spot and oil stain". The labeling tools used can be Labelme, LabelImg, etc. Finally, the particleboard sample image data is resized to a feature map of size 1280×1280×3 in the input layer model.

[0178] The present invention provides a method for detecting surface defects of particleboard. The method comprises the following steps: determining a multi-scale feature map of a sample to be detected corresponding to a particleboard with defects, anchoring the feature map of the sample to be detected with a corresponding prior frame group using a defect detection model, determining a prediction frame corresponding to each prior frame, and a prediction vector of the prediction frame. The prediction frame is screened using non-maximum suppression, and based on the prediction vector corresponding to the screened prediction frame and the prototype mask determined by dividing the feature map of the sample to be detected with the largest scale, a mask matrix corresponding to each target to be detected is determined. The defects in the particleboard are accurately segmented using the mask matrix. The present invention obtains an information-rich multi-scale feature map of the sample to be detected by using a feature extraction network and a feature fusion network. The segmentation layer of the defect detection model is used to determine the prediction vector corresponding to each prediction frame. The mask matrix is determined based on the mask coefficients in the prediction vector and the prototype mask. The mask matrix is used to accurately classify and segment the defects in the particleboard. The method has high computational efficiency and is more accurate in defect identification, framing, and segmentation. The feature extraction network and feature aggregation network constructed using an improved YOLOv5s algorithm reduce the number of model parameters and the amount of computation.

[0179] In order to more clearly illustrate the method for detecting surface defects of particleboard provided by the embodiment of the present invention, a specific example is given below.

[0180] Figure 7 is a schematic diagram of the training phase of the defect detection model provided by an embodiment of the present invention, such as Figure 7 As shown in the figure, the defect detection model consists of four parts: input layer, feature extraction network, feature aggregation network and segmentation layer.

[0181] The overall parameter information of the defect detection model is shown in Table 1, where -1 in the input column indicates the output from the previous layer; the module column indicates the name of the module; the number of modules indicates the actual number of modules used in the model; the tensor information column represents the parameter information of the module, such as the number of input channels, the number of output channels, the convolution kernel size, the stride, and the padding width; the output information column indicates the length, width, and number of output channels of the feature map output after the layer operation.

[0182] Table 1

[0183]

[0184]

[0185] (1) Input layer

[0186] The input layer of the defect detection model consists of sample particleboard surface image data and defect annotation information. Specifically, two industrial line scan cameras with a resolution of 4096x2 are used on the upper and lower surfaces of the particleboard production line to obtain complete particleboard surface image data. Defective particleboard images are selected and used as sample images.

[0187] Defect annotation information data: The defect area in the sample image is cropped to a fixed pixel size, for example, 1024*2048 pixels. The cropped sample image data is then augmented by adding Gaussian noise, vertical flipping, left-right rotation, median filtering, and random 90° rotation. This is done to improve the model's generalization ability, reduce overfitting, and minimize the impact of additional factors on the recognition results.

[0188] After preprocessing, the sample images were labeled and saved using tools like Labelme. The collected particleboard surface defects included shavings, dust, glue spots, oil stains, and core leakage. Finally, the input layer model resized the particleboard sample image data into a 1280×1280×3 feature map.

[0189] (2) Feature extraction network

[0190] The feature extraction network mainly consists of a convolutional Conv module, a Ghost Bottleneck module, a coordinate attention mechanism (Coord Att) module, and a simplified spatial pyramid pooling-Fast (simSPPF) module.

[0191] like Figure 7As shown in the figure, the feature extraction network consists of 13 layers: ① Conv modules are used in the 1st, 2nd, and 10th layers, which implement downsampling without losing particleboard image information. ② Ghost Bottleneck modules are used in the 3rd, 4th, 5th, 7th, 8th, and 11th layers, with downsampling also performed in the 4th and 7th layers. The Ghost Bottleneck module consists of convolutions with different kernels. It fuses information from before and after the layer through feature concatenation and exchanges image features from different channels without changing the size of the feature map, thus reducing the loss of object information caused by downsampling. At the same time, the Ghost Bottleneck module has fewer parameters and has a minimal impact on computational accuracy, effectively reducing the model's computational complexity. ③ The Coord Att module is used in the 6th, 9th, and 12th layers. This allows the network to focus more on the particleboard defects being detected, improving the detection of similar defects such as glue spots and oil stains. ④ The simSPPF module is used in the final layer of the network. This module aggregates features from layers of different sizes and connects them to the original feature map to obtain feature maps of different resolutions. This module integrates more features of different resolutions and obtains more information about surface defects. After five downsampling operations in the feature extraction network, the output feature map size becomes 40×40×512, which is the first feature map with the smallest partitioning scale.

[0192] (3) Feature Aggregation Network

[0193] The feature aggregation network mainly consists of a depth wise Conv (DWConv) module, a GhostBottleneck module, an upsampling module and a splicing module.

[0194] like Figure 7 As shown in Figure 2, the structure of the feature aggregation network includes a total of 16 layers of modules. Assuming that the DWConv module output from the simSPPF module in the feature extraction network is the first layer module, the process of achieving feature fusion of particleboard image data of different scales includes:

[0195] ① Use the DWConv module in the 1st, 4th, 6th, and 9th layers. The structure of the DWConv module is consistent with that of the Conv module, but each convolution kernel of the Conv module performs convolution operations on each channel. Compared with convolution operations on one channel per convolution kernel, the DWConv module can reduce the number of model parameters and reduce operating costs.

[0196] ② Use Ghost Bottleneck modules in the 5th, 10th, 13th, and 16th layers to utilize the deep network structure to obtain more information and further reduce the number of model parameters;

[0197] ③ Aggregate the 6th layer output of the feature extraction network with the 7th layer output of the feature aggregation network, and the 9th layer output of the feature extraction network with the 2nd layer output of the feature aggregation network. Use the splicing operation to upsample the features in combination with the feature information obtained by the feature extraction network. By transferring high-level features and supplementing low-level semantic information, high-level features with strong semantics can be obtained in the underlying network with high resolution, thereby improving the detection effect of small defects.

[0198] ④ Feature aggregation is performed on the 15th layer by combining the outputs of the 1st and 14th layers, and on the 12th layer by combining the outputs of the 6th and 11th layers. Downsampling is performed using a concatenation operation to compensate for and enhance positional information, retain more shallow positional features, and further enhance the network's ability to extract defect features. After the feature aggregation network operation, three Ghost Bottleneck modules in the 10th, 13th, and 16th layers of the network output feature maps of three different scales: 160×160×128, 80×80×256, and 40×40×512.

[0199] (4) Segmentation layer

[0200] The segmentation layer is mainly composed of Conv2d module, Conv module and upsampling module, and its main functions are composed of two parts, including Conv2d module calculating the prediction box output and Proto layer calculating the prototype mask.

[0201] ①Conv2d module prediction box output calculation process

[0202] After the sample image is processed by the feature extraction layer and the feature aggregation layer at the input layer, three different scales of the sample feature maps to be detected are output, with scales of 160×160×128, 80×80×256, and 40×40×512. Subsequently, the sample image is grid-divided based on different scales, that is, the 1280×1280 pixel sample image input of the input layer is divided into three grids with different receptive fields of 160×160, 80×80, and 40×40. The pixels represented by each grid in the three receptive fields are 8×8, 16×16, and 32×32, respectively. In the segmentation layer, three Conv2d modules are respectively operated to expand the output channels. The formula for calculating the number of channels after expansion is:

[0203] n=n c +n o +n m

[0204] Where n c Indicates the number of defect identification categories; n oThe dimension of the prediction vector corresponding to the model prediction box includes the prediction box confidence, the horizontal coordinate of the prediction box center point, the vertical coordinate of the prediction box center point, the prediction box width, and the prediction box height, i.e., no = 5; nm represents the dimension of the prediction mask coefficient of the defect detection model, which is usually 32. In the present invention, the defect categories mainly include wood shavings, dust, glue spots and oil stains, and core leakage, i.e., nc = 4.

[0205] The defect detection model also presets three sets of anchor boxes (three in each set, for a total of nine) with different aspect ratios for target prediction in each of the three output layers: [10×13, 16×30, 33×23], [30×61, 62×45, 59×119], and [116×90, 156×198, 373×326]. These three sets of anchor boxes predict targets within each grid cell of the corresponding receptive field: the [10×13, 16×30, 33×23] anchor boxes predict small defects in a 160×160 grid; the [30×61, 62×45, 59×119] anchor boxes predict medium defects in an 80×80 grid; and the [116×90, 156×198, 373×326] anchor boxes predict large defects in a 40×40 grid. Therefore, the model generates three prediction boxes within each grid in the three receptive fields, for a total of 3 × (160 × 160 + 80 × 80 + × 40 × 40) = 100,800 prediction boxes. Each prediction box contains the aforementioned n information. Finally, after the three Conv2d modules are run, the resulting tensor is [1, 100,800, n]. This means that the three Conv2d modules in the segmentation layer of the defect detection model ultimately output a set of 100,800 n-dimensional vectors. Each vector contains the following information:

[0206]

[0207] The physical meaning of each parameter in the prediction vector is the same as described above.

[0208] ②Proto layer prototype mask calculation process

[0209] The output of the Conv2d module corresponding to the feature map of the sample to be detected with the largest division scale in the segmentation layer is used as the input of the Proto layer network branch. After convolution processing and upsampling operations in the convolution layer, the mask tensor P with a dimension of [320, 320, 32] is output. That is, the Proto layer generates 32 prototype masks of size 320×320.

[0210] At the same time, take the mask coefficient P of the prediction box output [100800,32] cBy performing a linear combination operation with the prototype mask P, we can get the instance mask prediction result M corresponding to each prediction box. The calculation formula is:

[0211] M=σ(PP c T )

[0212] Where M represents the instance mask prediction result; σ represents the Sigmoid activation function, T represents the matrix transpose operation; P represents the prototype mask; P c Represents the mask coefficient of any predicted box.

[0213] The total loss function of the defect detection model has four parts, including classification loss (L class , classificationloss), confidence loss (L confidence ,confidence loss), segmentation loss (L mask , mask loss) and bounding box regression loss (L CIoU , boundary box positioning loss).

[0214] The confidence loss calculation formula is:

[0215]

[0216] Where, They represent the true confidence of the target to be detected in the j-th prior box of the i-th grid; Indicates that the j-th prior box of the i-th grid has a target to be detected, that is, a positive sample, otherwise it is 0; It means that there is no target to be predicted in the j-th prior box of the i-th grid, that is, a negative sample, otherwise it is 0. noobj is a weight value, which indicates the weight of the confidence error in the loss function when the prediction box does not predict the target. B represents the total number of prediction boxes included in the i-th grid; s 2 It means that the feature map of the sample to be detected is divided into s*s grids at the segmentation layer, that is, the specific scale of the grid.

[0217] The classification loss calculation formula is:

[0218]

[0219] Where, L class represents the classification loss; ρ i,j (r) represents the predicted probability value that the j-th prediction box of the i-th grid belongs to defect category r; It represents the true probability that the real box corresponding to the j-th prediction box of the i-th grid belongs to the defect category r.

[0220] The segmentation loss calculation formula is:

[0221]

[0222] Where M gt represents the true value of the instance mask, yz, They represent the predicted value and true value of the z-th pixel in the mask area within the j-th prediction box of the i-th grid, and Z represents the total number of all pixels.

[0223] The bounding box regression loss is calculated as:

[0224]

[0225] The meaning of each parameter in the formula has been explained before and will not be repeated here.

[0226] The total loss function can be expressed as: L = L confidence +L class +L mask +L CIoU ;

[0227] When the total loss function is minimized, the defect detection model training is completed, resulting in a trained defect detection model. Ideally, this is the case, where the total loss function can be minimized. In practical applications, this can be determined by determining whether the number of training cycles meets a preset training cycle threshold and whether the total loss function of the defect detection model has converged. When either the preset training cycle threshold or the total loss function converges, the defect detection model training is considered complete.

[0228] Figure 8 FIG. 1 is a schematic diagram of an implementation process of particleboard surface defect detection provided by an embodiment of the present invention, such as Figure 8 As shown in the figure, the defect detection model detects surface defects on particleboard. The implementation process of the input layer, feature extraction network, and feature aggregation network is the same as described above. The main difference is that the non-maximum suppression (NMS) algorithm is used in the segmentation layer to filter the prediction vectors corresponding to the prediction boxes, filter out redundant prediction boxes, and output the index value with the maximum prediction confidence for each prediction box, ultimately obtaining multiple (assuming it is represented by x) prediction bounding boxes. The output information of the prediction bounding box is a feature vector of dimension [x, 38], where 38 specifically refers to: defect confidence score (dimension: 1), defect prediction box location (dimension: 4), maximum defect category probability index value (dimension: 1), and defect segmentation mask coefficient (dimension: 32).

[0229] At the same time, the instance mask result is obtained by linearly combining the mask coefficients in the prediction vector (dimension [x, 32]) with the prototype mask result of the Proto layer. This is then cropped and binarized to obtain the final predicted segmentation result. Cropping refers to clearing the mask values outside the predicted bounding box to zero, and binarization refers to binarizing the generated instance mask with a threshold of 0.5. This results in more accurate predictions of particleboard defect boundaries and more accurate identification of the defect types and categories corresponding to surface defects.

[0230] In addition, the present invention also uses model accuracy evaluation indicators and model scale evaluation indicators to evaluate the model detection effect of the defect detection model provided by the present invention. The model accuracy evaluation indicators include: ① Precision (Precious); ② Recall (recall); ③ Mean Average Precious (mAP) of mAP@0.5 and mAP@0.5:0.95, where mAP@0.5 is the prediction result when the threshold of the prediction box and the true value box is set to 0.5; mAP@0.5:0.95 is the prediction result obtained by calculating an mAP every 0.05 from 0.5 to 0.95 and averaging it; ④ Total model training loss. Model scale evaluation indicators include: model training time, number of model parameters, model weight file size, and model floating-point calculation amount.

[0231] And through simulation means, the YOLOv5s model and the defect detection model proposed in the present invention were used to perform training and recognition results comparison respectively. The original sample data were 496 images of four defects including wood shavings, dust, glue spots and oil stains on the surface of particleboard collected from a factory over three months (the glue block, glue spot and oil stain defect images with similar characteristics were merged into one defect type of "glue spot and oil stain") and leaking cores, of which 137 were wood shavings defects, 83 were dust spot defects, 223 were glue spots and oil stains defects, and 53 were leaking cores defects. After image preprocessing, the sample images were expanded to 2976 and data annotation was performed. The annotated data were used to train the two models respectively. The hyperparameter settings of the YOLOv5s model and the defect detection model proposed in the present invention during training are shown in Table 2. As the number of training times increases, the change curves of the accuracy of the YOLOv5s model and the defect detection model proposed in the present invention are shown as follows: Figure 9 As shown in the figure, the horizontal axis represents the number of training times, the vertical axis represents the accuracy rate, the solid line represents the accuracy rate change curve corresponding to the YOLOv5s model, and the dotted line represents the accuracy rate change curve corresponding to the defect detection model proposed in the present invention. Similarly, the YOLOv5s model and the defect detection model proposed in the present invention correspond to the change curves of the recall rate as shown in the figure. Figure 10As shown in the figure, the YOLOv5s model and the defect detection model proposed in this invention correspond to the change curves of mean average precision mAP@0.5 and mean average precision mAP@0.5:0.95 respectively. Figure 11 and Figure 12 As shown in the figure, the YOLOv5s model and the defect detection model proposed in this invention correspond to the change curves of the total loss of model training respectively. Figure 13 shown.

[0232] Finally, the comparison results of model evaluation indicators are shown in Tables 3 and 4.

[0233] Table 2 Model hyperparameter settings

[0234] Initial learning rate momentum Attenuation coefficient Number of warm-up rounds Batch size Number of training rounds 0.01 0.937 0.0005 3 16 300

[0235] Table 3 Comparison of model accuracy evaluation results

[0236] Model Accuracy Recall Average precision mAP@0.5 Average precision mAP@0.5: 0.95 YOLOv5s 0.908 0.906 0.929 0.629 Defect detection model 0.921 0.897 0.916 0.609

[0237] Table 4 Comparison of model scale evaluation results

[0238]

[0239] like Figure 9 、 Figure 10 、 Figure 11 、 Figure 12 and Figure 13 As shown in the figure and the results in Tables 3 and 4, the defect detection model proposed in this paper significantly reduces the model computational complexity compared to the YOLOv5 model, while significantly improving the accuracy of defect classification prediction and demarcation, as well as the accuracy of segmentation. Specifically, the training time of the defect detection model proposed in this paper was reduced by approximately 3.58%, the number of model parameters decreased by approximately 63.5%, the model weight file size decreased by approximately 60.54%, and the model floating-point computation complexity decreased by approximately 65.2%. These comparative results further demonstrate the effectiveness of the defect detection model proposed in this paper.

[0240] The defect detection model proposed in this invention is used to detect defects on the sample images of particleboard surfaces. The corresponding detection results include the defect category, defect confidence, defect border, and defect boundary. The corresponding detection result diagram is shown in the figure below. Figure 14 As shown, Figure 14The rectangular border in the middle represents the prediction box corresponding to different defect types. The pattern with a teardrop-shaped outline border indicates the defect type is dust, with a confidence level of 0.79; the pattern with a circular outline border indicates the defect type is glue spots and oil stains, with a confidence level of 0.91; the pattern with a triangular outline border indicates the defect type is sand leakage, with a confidence level of 0.82; and the pattern with an elliptical outline border indicates the defect type is wood particles, with a confidence level of 0.99. In order to more clearly illustrate the different types of defects on the particleboard surface, Figure 14 The display results of different defects in the particleboard are magnified. In actual application, the defects of different types on the surface of particleboard may vary in size. Generally, they are larger than Figure 14 Of course, different characters can also be used to represent different defect types, such as "bh" for wood shavings, "jbyw" for glue spots and oil stains, "fc" for dust defects, and "ls" for core leaks.

[0241] On the particleboard production line, when defective boards are identified, they are linked with the on-site panel arrangement PLC controller to complete the on-site panel arrangement operation.

[0242] Figure 15 FIG. 1 is a schematic diagram of the structure of a device for detecting surface defects of particleboard provided by an embodiment of the present invention, as shown in FIG. Figure 15 As shown, the device includes:

[0243] Feature map module 1501 is used to determine a feature map of a sample to be detected corresponding to each sample image based on the feature extraction network and feature fusion network in the defect detection model; the sample image is an image of particleboard with defects; the feature map of the sample to be detected includes an image divided into grids according to three scales;

[0244] The prediction frame module 1502 is used to determine the prediction frame corresponding to the prior frame in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction frame based on the segmentation layer in the defect detection model; the prediction vector includes a mask coefficient used to represent the weight corresponding to each target to be detected in the prediction frame;

[0245] Extraction module 1503, configured to extract a prototype mask corresponding to a reference sample feature map through upsampling and multi-layer convolution operations; the reference sample feature map is the one with the largest division scale in the feature map of the sample to be detected;

[0246] Determination module 1504 is configured to determine a mask matrix corresponding to the target to be detected within each prediction frame based on the mask coefficients in the prediction vector corresponding to the reference prediction frame and the prototype mask corresponding to the reference sample feature map; the reference prediction frame is obtained by screening the prediction frame within each grid in the target sample feature map based on a non-maximum suppression algorithm; the mask matrix is used to represent the predicted segmentation region corresponding to the target to be detected;

[0247] The segmentation module 1505 is configured to perform cropping and binarization processing on the feature map of the sample to be detected based on the mask matrix to obtain a segmentation result of the sample image; the segmentation result is the boundary of each target to be detected in the sample image;

[0248] The defect detection model is used to anchor according to the prior frame group corresponding to the feature map of the sample to be detected at different scales, and determine the prediction frame closest to the target to be detected in the feature map of the sample to be detected, the confidence, the defect category to which the target to be detected belongs, and the boundary of the target to be detected.

[0249] The device for detecting surface defects of particleboards provided in an embodiment of the present invention can execute the technical solution of the method for detecting surface defects of particleboards in any of the above-mentioned embodiments. Its implementation principle and beneficial effects are similar to the implementation principle and beneficial effects of the method for detecting surface defects of particleboards. Please refer to the implementation principle and beneficial effects of the method for detecting surface defects of particleboards, and no further details will be given here.

[0250] Figure 16 FIG is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 16 As shown, the electronic device may include: a processor 1610, a communication interface 1620, a memory 1630, and a communication bus 1640, wherein the processor 1610, the communication interface 1620, and the memory 1630 communicate with each other via the communication bus 1640. The processor 1610 may call the logic instructions in the memory 1630 to execute a method for detecting surface defects of particleboard, which includes:

[0251] Based on the feature extraction network and feature fusion network in the defect detection model, a feature map of a sample to be detected corresponding to each sample image is determined; the sample image is an image of particleboard with defects; the feature map of the sample to be detected includes an image divided into grids according to three scales;

[0252] Based on the segmentation layer in the defect detection model, determine the prediction box corresponding to the prior box in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction box; the prediction vector includes a mask coefficient for representing the weight corresponding to each target to be detected in the prediction box;

[0253] Extracting a prototype mask corresponding to a reference sample feature map through upsampling and multi-layer convolution operations; the reference sample feature map is the one with the largest division scale in the feature map of the sample to be detected;

[0254] Determine a mask matrix corresponding to the target to be detected within each prediction frame based on the mask coefficients in the prediction vector corresponding to the reference prediction frame and the prototype mask corresponding to the reference sample feature map; the reference prediction frame is obtained by screening the prediction frame within each grid in the feature map of the sample to be detected based on a non-maximum suppression algorithm; the mask matrix is used to represent the predicted segmentation area corresponding to the target to be detected;

[0255] Based on the mask matrix, the feature map of the sample to be detected is cropped and binarized to obtain a segmentation result of the sample image; the segmentation result is the boundary of each target to be detected in the sample image;

[0256] The defect detection model is used to anchor according to the prior frame group corresponding to the feature map of the sample to be detected at different scales, and determine the prediction frame closest to the target to be detected in the feature map of the sample to be detected, the confidence, the defect category to which the target to be detected belongs, and the boundary of the target to be detected.

[0257] In addition, the logic instructions in the above-mentioned memory 1630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0258] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the methods for detecting particleboard surface defects provided by the above methods.

[0259] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is configured to execute the particleboard surface defect detection method provided by the above methods when the computer program is executed by a processor.

[0260] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0261] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0262] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting surface defects of particleboard, characterized in that: include: Based on the feature extraction network and feature fusion network in the defect detection model, a feature map of a sample to be detected corresponding to each sample image is determined; the sample image is an image of particleboard with defects; the feature map of the sample to be detected includes an image divided into grids according to three scales; Based on the segmentation layer in the defect detection model, determine the prediction box corresponding to the prior box in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction box; the prediction vector includes a mask coefficient for representing the weight corresponding to each target to be detected in the prediction box; Through upsampling and multi-layer convolution operations, the prototype mask corresponding to the reference sample feature map is extracted; The reference sample feature map is the one with the largest division scale in the feature map of the sample to be detected; Determine a mask matrix corresponding to an object to be detected in each prediction frame based on a mask coefficient in a prediction vector corresponding to a reference prediction frame and a prototype mask corresponding to the reference sample feature map; The reference prediction frame is obtained by screening the prediction frame in each grid in the feature map of the sample to be detected based on the non-maximum suppression algorithm; The mask matrix is used to represent the predicted segmentation area corresponding to the target to be detected; Based on the mask matrix, the feature map of the sample to be detected is cropped and binarized to obtain a segmentation result of the sample image; the segmentation result is the boundary of each target to be detected in the sample image; The defect detection model is used to anchor according to the prior frame group corresponding to the feature map of the sample to be detected at different scales, and determine the prediction frame closest to the target to be detected in the feature map of the sample to be detected, the confidence level, the defect category of the target to be detected, and the boundary of the target to be detected; The method of determining a feature map of a sample to be detected corresponding to each sample image based on a feature extraction network and a feature fusion network in the defect detection model includes: The feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample image to obtain first feature maps of three different division scales; Based on a deep convolutional network, feature aggregation is performed on the concatenation of the upsampled second feature map and the third feature map to determine a first aggregation result; the second feature map is the result of convolution processing on the smallest-scale segmentation in the first feature map; the third feature map is the second-smallest-scale segmentation in the first feature map; Based on a shallow convolutional network, feature aggregation is performed on the concatenation of the upsampled first aggregation result and the fourth feature map, a second aggregation result is determined, and the feature map of the sample to be detected with the largest division scale is output to the segmentation layer; the fourth feature map is the one with the largest division scale in the first feature map; Performing convolution processing on the concatenation result of the second aggregation result and the first aggregation result to determine a third aggregation result, and outputting the third aggregation result as the feature map of the sample to be detected with the second largest division scale to the segmentation layer; A convolution process is performed on the splicing result of the third aggregation result and the second feature map, and the feature map of the sample to be detected with the smallest division scale is output to the segmentation layer.

2. The method for detecting surface defects of particleboard according to claim 1, characterized in that: The training method of the defect detection model includes: Based on the segmentation layer in the defect detection model, determine the positive and negative sample labels and prediction vectors corresponding to each prediction box; the prediction box is the prediction box corresponding to each prior box in each grid in the feature map of the sample to be detected; the positive and negative samples are used to indicate whether the target to be detected exists in the prediction box; Determine the classification loss corresponding to the feature map of the sample to be detected based on a first predicted probability, a first actual probability, and positive and negative sample labeling parameters; the first predicted probability is the probability that the target to be detected in the predicted box belongs to the first defect category; the first defect category is any one of the defect categories; the first actual probability is the true probability that the target to be detected in the prior box corresponding to the prediction box to which the first predicted probability belongs belongs to the first defect category; Determine the confidence loss corresponding to the feature map of the sample to be detected based on the prediction confidence in the prediction vector corresponding to the prediction box, the true confidence corresponding to the prior box corresponding to the prediction box, and the positive and negative sample labeling parameters; Determine the segmentation loss corresponding to the feature map of the sample to be detected based on a first predicted mask matrix and a first actual mask matrix, as well as positive and negative sample labeling parameters; the first predicted mask matrix is a result of a linear combination of the mask coefficients corresponding to the predicted box and the prototype mask corresponding to the reference sample feature map; the first actual mask matrix is a real mask area corresponding to the grid to which the predicted box belongs; Determine the bounding box regression loss corresponding to the feature map of the sample to be detected based on the position of the predicted box in the prediction vector corresponding to the predicted box and the position of the real box that overlaps with the predicted box; Determine the bounding box regression loss corresponding to the feature map of the sample to be detected based on the intersection-over-union ratio of the predicted box and the true box corresponding to the target to be detected in the grid, the distance between the center point of the predicted box and the center point of the true box, the similarity between the aspect ratio of the predicted box and the aspect ratio of the true box, and the weight coefficient corresponding to the similarity; The training of the defect detection model is completed when the total loss function determined by the classification loss, the confidence loss, the segmentation loss and the bounding box regression loss is minimized.

3. The method for detecting surface defects of particleboard according to claim 1, characterized in that: The feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample feature map to obtain three first feature maps with different division scales, including: Based on the multi-layer convolutional network in the feature extraction network, downsampling the labeled sample image to obtain the sample images at different sampling rates; Based on the coordinate attention mechanism, the correlation relationship between the position information and channels of the sample image at different sampling rates is established to obtain the first feature maps of three different division scales; Based on the simplified spatial pyramid pooling layer, the first feature maps of the three different division scales are converted into feature maps of a fixed scale, and the first feature map with the smallest division scale is updated.

4. The method for detecting surface defects of particleboard according to claim 1, characterized in that: The feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample feature map to obtain the first feature maps of three different division scales, including: Performing a preprocessing operation on the sample image; the preprocessing operation includes: uniform pixel cropping and sample expansion; The defects in the pre-processed sample image are annotated by an annotation tool; the annotation includes annotating the defect category, the real mask area corresponding to the defect, and the real confidence level corresponding to the defect.

5. The method for detecting surface defects of particleboard according to claim 1, characterized in that: The prediction vector also includes prediction confidence, the position of the prediction box and the predicted defect category, wherein the position of the prediction box is represented by the horizontal coordinate, vertical coordinate, width and height of the center point of the prediction box; the predicted defect category is used to represent the characteristic value of the target to be detected in the prediction box belonging to each defect category.

6. The method for detecting surface defects of particleboard according to claim 5, characterized in that: The determining, based on a position of a prediction box in a prediction vector corresponding to the prediction box and a position of a real box overlapping with the prediction box, a bounding box regression loss corresponding to the feature map of the sample to be detected, includes: Based on the position of the predicted frame and the position of the real frame, respectively determining the intersection-over-union ratio of the predicted frame and the real frame, the distance between the center point of the predicted frame and the center point of the real frame, and the similarity between the aspect ratio of the predicted frame and the aspect ratio of the real frame; Determining a weight coefficient corresponding to the similarity based on the similarity and the intersection-over-union ratio; Determine the diagonal distance of the minimum rectangular box including the predicted box and the true box as the diagonal distance; Determining a regression loss corresponding to a single prediction box based on the intersection-over-union ratio, the distance between the center points, the diagonal distance, the similarity, and a weight coefficient corresponding to the similarity; Based on all corresponding prediction boxes in the feature map of the sample to be detected and the regression loss corresponding to a single prediction box, the regression loss of the bounding box corresponding to the feature map of the sample to be detected is determined.

7. The method for detecting surface defects of particleboard according to claim 2, characterized in that: The first prediction probability is a result of processing the characteristic value of the predicted defect category in the prediction vector corresponding to the prediction frame using a normalized exponential function.

8. The method for detecting surface defects of particleboard according to claim 2, characterized in that: The classification loss, the confidence loss, and the segmentation loss are determined based on a binary cross entropy loss function.

9. A device for detecting surface defects of particleboard, characterized in that: include: A feature map module is configured to determine a feature map of a sample to be detected corresponding to each sample image based on a feature extraction network and a feature fusion network in a defect detection model; the sample image is an image of particleboard with defects; and the feature map of the sample to be detected includes an image divided into grids according to three scales; A prediction frame module is used to determine, based on the segmentation layer in the defect detection model, the prediction frame corresponding to the prior frame in each grid in the feature map of the sample to be detected, and the prediction vector corresponding to the prediction frame; the prediction vector includes a mask coefficient used to represent the weight corresponding to each target to be detected in the prediction frame; The extraction module is used to extract the prototype mask corresponding to the reference sample feature map through upsampling and multi-layer convolution operations; The reference sample feature map is the one with the largest division scale in the feature map of the sample to be detected; A determination module, configured to determine a mask matrix corresponding to an object to be detected in each prediction frame based on a mask coefficient in a prediction vector corresponding to a reference prediction frame and a prototype mask corresponding to the reference sample feature map; The reference prediction frame is obtained by screening the prediction frame in each grid in the feature map of the sample to be detected based on the non-maximum suppression algorithm; The mask matrix is used to represent the predicted segmentation area corresponding to the target to be detected; A segmentation module, configured to perform cropping and binarization processing on the feature map of the sample to be detected based on the mask matrix to obtain a segmentation result of the sample image; the segmentation result is the boundary of each target to be detected in the sample image; The defect detection model is used to anchor according to the prior frame group corresponding to the feature map of the sample to be detected at different scales, and determine the prediction frame closest to the target to be detected in the feature map of the sample to be detected, the confidence level, the defect category to which the target to be detected belongs, and the boundary of the target to be detected; The method of determining a feature map of a sample to be detected corresponding to each sample image based on a feature extraction network and a feature fusion network in the defect detection model includes: The feature extraction network constructed based on the improved YOLOv5s algorithm processes the sample image to obtain first feature maps of three different division scales; Based on a deep convolutional network, feature aggregation is performed on the concatenation of the upsampled second feature map and the third feature map to determine a first aggregation result; the second feature map is the result of convolution processing on the smallest-scale segmentation in the first feature map; the third feature map is the second-smallest-scale segmentation in the first feature map; Based on a shallow convolutional network, feature aggregation is performed on the concatenation of the upsampled first aggregation result and the fourth feature map, a second aggregation result is determined, and the feature map of the sample to be detected with the largest division scale is output to the segmentation layer; the fourth feature map is the one with the largest division scale in the first feature map; Performing convolution processing on the concatenation result of the second aggregation result and the first aggregation result to determine a third aggregation result, and outputting the third aggregation result as the feature map of the sample to be detected with the second largest division scale to the segmentation layer; A convolution process is performed on the splicing result of the third aggregation result and the second feature map, and the feature map of the sample to be detected with the smallest division scale is output to the segmentation layer.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting surface defects of particle board according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting surface defects of particle board according to any one of claims 1 to 8 is implemented.

Images