Eucalyptus veneer defect image detection system and detection method

Abstract

A eucalyptus veneer defect image detection system and method are proposed to achieve eucalyptus veneer defect identification. The defect category identification is balanced using the Bbox-cover method. The designed aggregation module AGM is used to effectively fuse YOLOv5 channel information and pixel information, thereby improving the accuracy of eucalyptus defect detection. Finally, a defect area calculation method is designed to meet the user's needs for screening and counting defects of different sizes.

Description

Technical field:

[0001] The invention discloses a eucalyptus veneer defect image detection system and a detection method, belonging to the technical field of intelligent identification of wood surface pit detection. Background technology:

[0002] Eucalyptus veneer is the raw material for everyday wooden products, such as desks and bedside cabinets. Eucalyptus veneer can contain various defects, including dead knots, loose knots, bark, and cracks, which can affect the final appearance and quality of the wood product. According to research, defect sorting of eucalyptus veneer is required during the eucalyptus board production process, and this task is mostly performed manually. With the advancement of computer technology, many researchers are currently using fully supervised learning to identify and locate wood defects:

[0003] Chinese patent document CN113066079A discloses a method, system, and storage medium for automatic wood defect detection, which includes: obtaining an original scanned image of a wood board; performing background difference processing on the original scanned image to remove the background area from the original scanned image to obtain a wood board area image; adjusting the image size of the wood board area image according to a predetermined size to obtain an image to be detected; after performing brightness adaptive enhancement on the image to be detected, inputting the image to be detected into a detection and analysis deep learning model to obtain detected defect information; locating the position coordinates of each defect in the wood board area image based on the defect information, and integrating the defect information and outputting it. The present invention can complete the classification and screening of wood without relying on human observation; its recognition efficiency is high, which is conducive to promotion in large-scale production automation.

[0004] Chinese patent document CN113160136A discloses a wood defect recognition and segmentation method based on an improved Mask R-CNN, the process of which is as follows: 1. Collecting a data set; 2. Balancing the data set samples; 3. Expanding the number of data sets; 4. Dividing the data set and making it into the required format; 5. Selecting the structure of the detection network; 6. Inputting the data set into the network model, using the Momentum optimization method, and training the network 20,000 times; 7. After every 1,000 training times, the model tests the validation set, and selects the model with the highest detection accuracy among the 20 times as the trained detection model; 8. Using the trained model to test the test set, the accuracy and recall rate of the model and the detection effect diagram of the model are obtained; the present invention uses artificial intelligence methods to simultaneously complete defect classification, defect location detection and defect contour segmentation, and is suitable for the rapid processing of wood defects in wood industry processing lines.

[0005] Chinese patent document CN114359235A discloses a method for detecting wood surface defects based on an improved YOLOv51 network, including: obtaining a publicly available wood defect dataset as dataset 1; constructing an improved YOLOv51 network model; and training the improved YOLOv51 network model using dataset 1 to obtain a trained network model, model 1. Real-world wood data is collected as dataset 2, which is divided into a training set, a validation set, and a test set. Model 1 is used to predict the test set, labeling wood images with defects in the test set, and using labels greater than a set prediction threshold as pseudo-labels. Model 1 is then trained by fine-tuning the training and test sets in dataset 2 to obtain model 12. The validation set is used to select the optimal network model from model 1 and model 2. Wood surface defects are detected using the optimal network model. The present invention is capable of detecting various types of wood surface defects, effectively improving recognition accuracy and efficiency.

[0006] The method described in the aforementioned patent document requires a high volume of experimental data, and the sample sizes of the captured image defects are extremely unbalanced. This requires a highly stable device to capture real images from the actual production process and address the sample imbalance. Therefore, a system for image acquisition and sorting eucalyptus veneer is urgently needed. Summary of the invention:

[0007] In view of the deficiencies in the prior art, the present invention discloses a eucalyptus veneer defect image detection system.

[0008] The invention also discloses a detection method based on the detection system. SUMMARY OF THE INVENTION

[0010] The eucalyptus veneer defect image detection method and system described in the present invention realize eucalyptus veneer defect identification, wherein the identification of the defect category is balanced using the Bbox (Boundingbox)-cover method, and the designed aggregation module AGM is used to effectively fuse the channel information and pixel information of YOLOv5, thereby improving the accuracy of eucalyptus defect detection. Finally, a defect area calculation method is designed to meet the user's needs for screening and counting different defect sizes.

[0011] The detailed technical solutions of the present invention are as follows:

[0012] A eucalyptus veneer defect image detection system includes a high-speed board collecting device, and is characterized in that it also includes: an image acquisition module, a model training module and a model recognition module; the image acquisition module, model training module and model recognition module are installed in the high-speed board collecting device to collect, process, identify and detect the eucalyptus veneer entering the high-speed board collecting device.

[0013] Preferably, according to the present invention, the eucalyptus veneer defect image detection system further includes: a sorting device, wherein the sorting device controls the forward and reverse rotation of the motor through a PLC to thereby control the direction of the grading baffle, wherein the PLC controls the forward and reverse rotation of the motor according to the eucalyptus veneer defect image detection result, wherein the detection result includes first-level board and second-level board;

[0014] If it is detected as a first-level board, the motor is controlled to rotate forward, and the eucalyptus veneer falls into the first-level board bin;

[0015] If it is detected as a secondary board, the motor is controlled to reverse and the eucalyptus veneer falls into the secondary board bin; the distinction between the primary board and the secondary board is distinguished and identified by the model recognition module.

[0016] Preferably, according to the present invention, the PLC and the edge computing box adopt the Modbus-RTU protocol, and communicate through the 485 interface, with the edge computing box (T506) as the host and the PLC as the slave, wherein the register 40001 address is held as the status code, and a total of 5 types of status codes are designed, including 0 for successful data reading, 1 for writing statistical data, 2 for camera abnormality, 3 for other abnormalities, 4 for model initialization, and 5 for system preparation; 40002-40005 are respectively the number of four types of defects, 40011-40012 are respectively the dead knot and loose knot area settings, and 400015 is the image saving setting, and the four types of defects include dead knots, loose knots, bark and cracks; the model recognition module is loaded in the edge computing box.

[0017] A detection method based on the above detection system includes image acquisition, image training and image recognition, and is characterized in that:

[0018] In the image acquisition:

[0019] The Bbox-cover method is used to operate the collected original image dataset, as shown in the flowchart. Figure 3 , perform label balancing on each collected data image, the steps are as follows:

[0020] Step 1: Sequentially select an image from the dataset as the operation object F;

[0021] Step 2: Determine the label category of the operation object F to see if the operation object F has four types of defects q, including: q0 represents a dead knot, q1 represents a slip knot, q2 represents bark, and q3 represents a crack;

[0022] If a certain defect category is missing, perform the above Copy-paste operation on the operation object F:

[0023] The operation object F is used as the main image I main , for the missing defect category q i Copy to the operation object F, where i∈(0, 1, 2, 3), and go to Step 3;

[0024] Step 3: Count the defect labels and calculate the mean number of defect categories In the process of i and Perform absolute value calculation to obtain q i- or q i+ ,q i- Represents the defect category that needs to be reduced, q i+ Represents the defect categories that need to be added among the four types of defects;

[0025] Get q i+ Defect image Bbox1 coordinate point (x min ,y min , x max ,y max ) to crop and cover q i- The coordinate point (x′ min , y′ min , x′ max , y′ max ) until the number a is met, where the Bbox coordinate points represent the upper left and lower right corner coordinate points of the target box after normalization, and β is the proportional coefficient:

[0026]

[0027]

[0028] The value of β is 0-1;

[0029] Step 4: Complete the above steps from Step 1 to Step 3 for each image in the entire dataset to achieve a balance in the defect categories of the entire dataset;

[0030] In the image training:

[0031] The defects of dead knots, loose knots, cracks and bark in eucalyptus veneer also have different channel dimension information in the image. Therefore, the attention mechanism module SE-Net (Sequeeze and Excitation Net) is added to the YOLOv5 structure. Its structure is as follows Figure 4-1 As shown, to obtain the feature layer of the input image, the weight of each channel, the specific steps are as follows:

[0032] First, the global average pooling is used to output 1×1×C data:

[0033]

[0034] Formula (IV) is also called Fsq (Squeeze) operation, which means compressed feature map operation in Chinese. H represents the image height, W represents the image width, and Fsq operation is performed on C feature maps of size H*W. c A two-dimensional matrix representing the channel information of each feature map, where the subscript c represents the number of channels of each feature map. Then comes the Fex (excitation) operation, which is the extraction feature map operation in Chinese. The formula is as follows (V), where w1 is multiplied by the operation result of each feature map obtained by formula (IV) to obtain (1×1×C) / r, which is the first fully connected layer. After passing through Relu, the output dimension remains unchanged. Then it is multiplied by w2, which is the second fully connected layer. The output dimension is 1×1×C, and δ is the Relu function. This operation passes through two fully connected layers and finally passes through the Sigmod function for data normalization to obtain the result s:

[0035] s=F ex (z, w) = σ(w2δ(w1z)) (V)

[0036] In formula (V), z is the operation result of each feature map obtained by formula (IV); w is the secondary full connection operation, w1 represents the first operation, and w2 represents the second operation; σ is the sigmoid operation;

[0037] Finally, through F scale The operation, which is a recalibration operation in Chinese, multiplies the results of the Fsq operation and the Fex operation:

[0038] F scale (u c , s c )=s c ·u c (VI)

[0039] In formula (VI), u c is a two-dimensional matrix of each feature map channel information; sc is the weight; through F scaleTo enhance important channel information, this technical feature uses SENet to obtain more features about the channel dimension of eucalyptus defects, ultimately forming a recognition model.

[0040] In the image recognition:

[0041] After the eucalyptus defect image information is identified by the recognition model shown, the P(x p ,y p ) and Q(x q ,y q ) The distance between two pixels D(P, Q):

[0042]

[0043] Then, using the actual measured length and width of the eucalyptus, the actual distance of each pixel is obtained through formula (VIII):

[0044]

[0045] In formula (VIII), d w is the actual width of the eucalyptus veneer; d l is the length of the eucalyptus veneer; D w is the distance between the pixels of the eucalyptus veneer width in the image; D l is the distance in pixels of the length of the eucalyptus board in the image; the image size is 2048*2048px, and it is calculated that 1 pixel is approximately 0.70mm. The defect image is identified by the recognition model, the ROI area is extracted, and the circumscribed rectangular frame of the defect is drawn using image processing. Finally, the area of ​​the rectangular frame is calculated to obtain the area of ​​the defect image.

[0046] According to the present invention, the missing defect category q i When copying to the operation object F, specifically:

[0047] First, traverse the label file to obtain the missing defect category label;

[0048] Secondly, the defect image corresponding to the missing defect category label is copied and pasted to a random position of the operation object F. At the same time, the position information of the random position is updated to the defect category label; until the operation object F meets the label judgment condition, enter Step 3.

[0049] According to the preferred embodiment of the present invention, before adopting the Bbox-cover method, the following operations are performed on the defect types such as slip knots, bark, and cracks with fewer defects using the Copy-paste strategy, and the steps are as follows:

[0050] Through screening, images with slip knots, bark, and cracks are selected as source images I src , used to extract the defect image and paste it into the main image I main Random position in the source image I src and the main image I main Perform random horizontal inversion respectively, or perform random horizontal inversion on the source image I src Perform random scale scaling; finally, use formula (III) for synthesis:

[0051] I=I src ×α+I main ×(1-α) (III)

[0052] In formula (III), α is the mask, i.e., the pixel area of ​​the defective portion of the source image; I represents the new image after copy-pasting. This technical solution partially enhances images with fewer defect categories using the aforementioned method. This involves first expanding the original image to include images with fewer categories, then using a Bbox-cover method to balance the defect categories within each image. Ultimately, this approach effectively addresses the imbalance in defect categories in eucalyptus wood.

[0053] According to the preferred embodiment of the present invention, the image training further includes an aggregation module AGM, such as Figure 4-2 As shown in the figure, the channel dimension information of the image is effectively combined with the pixel information, the input image channel dimension is maximized pooled, the pixel dimension information is averaged pooled, and finally spliced ​​and output to the detection head after completing the 1×1 convolution. The experiment found that with a small increase in the amount of calculation, the model map index increased by about 0.10. The overall modified network structure of YOLOv5 is shown in the figure. Figure 5 .

[0054] The technical advantages of the present invention are:

[0055] 1. The detection system and method of the present invention realizes real-time monitoring and real-scene acquisition of defect images of eucalyptus veneer, and can work stably and efficiently in actual production lines, with a maximum transmission speed of 70 meters per minute.

[0056] 2. This paper uses a self-designed Bbox-cover method to balance the collected categories, addressing the data imbalance of various eucalyptus defects in the original data. This ensures that the defect categories of each eucalyptus veneer are evenly distributed, ultimately enhancing the dataset to include over 5,000 eucalyptus defect images.

[0057] 3. The present invention also adopts a modified YOLOv5 structure and adds a SENet structure, so that the structure focuses on extracting information of the channel dimension of defects while suppressing unimportant information. The designed aggregation module (AGM) effectively fuses the sampled channel information with pixel information to improve the effect of small target defect detection. In the training results, the map@0.5 index can reach about 0.72, and the FPS is 68. Map represents the average AP value of all defect categories. The higher the value, the better the detection result; FPS represents the number of images that the model can infer per second.

[0058] 4. The present invention also designs a method for calculating the size of eucalyptus wood defect area. By using the result image recognized by the model, the defect area of ​​each type of wood is calculated and the number is counted to meet the defect size setting requirements set by the user.

[0059] 5. The system described in the present invention can also realize secondary classification of eucalyptus veneers, with a sorting speed of up to 50 sheets / minute, a sorting accuracy of about 90%, and a defect image recognition speed of 0.11s / sheet in the edge computing box. Description of the drawings:

[0060] Figure 1 is the eucalyptus veneer image to be classified and identified in the present invention;

[0061] Figure 2 It is a schematic diagram of the high-speed board collecting device in the present invention;

[0062] Figure 3 is a flow chart of the Bbox-cover method of the present invention;

[0063] Figure 4-1 It is a schematic diagram of the SE-Net network structure of the present invention;

[0064] Figure 4-2 is a schematic diagram of the AGM network structure of the present invention;

[0065] Figure 5 The improved YOLOv5 network structure diagram in the present invention;

[0066] Figure 6 Schematic diagram of the results of image recognition of eucalyptus veneer according to an embodiment of the present invention;

[0067] Figure 7 A schematic diagram of the process of image recognition and detection of eucalyptus veneer by the system of the present invention;

[0068] Figure 8 In the embodiment of the present invention, the results of the comparative experiment of the method of the present invention and other existing recognition algorithms are shown in the table;

[0069] Figure 9In the embodiment of the present invention, the effect table of the comparative experiment of the method of the present invention and other existing recognition algorithms is shown. Specific implementation method:

[0070] The present invention will be described in detail below with reference to the embodiments and the accompanying drawings, but is not limited thereto.

[0071] The present invention is used to perform model training set image recognition on a batch of eucalyptus veneers, wherein the eucalyptus veneers to be identified are as follows: Figure 1 The specific implementation process of the entire system is as shown in Figure 7 shown.

[0072] Example 1

[0073] like Figure 2 A eucalyptus veneer defect image detection system includes a high-speed board collector, an image acquisition module, a model training module, and a model recognition module. The image acquisition module, model training module, and model recognition module are installed in the high-speed board collector to acquire, process, identify, and detect eucalyptus veneers entering the high-speed board collector.

[0074] The eucalyptus veneer defect image detection system further includes: a sorting device, wherein the sorting device controls the forward and reverse rotation of a motor through a PLC to thereby control the direction of a grading baffle, wherein the PLC controls the forward and reverse rotation of the motor based on the eucalyptus veneer defect image detection results, wherein the detection results include first-level boards and second-level boards;

[0075] If it is detected as a first-level board, the motor is controlled to rotate forward, and the eucalyptus veneer falls into the first-level board bin;

[0076] If it is detected as a secondary board, the motor is controlled to reverse and the eucalyptus veneer falls into the secondary board bin; the distinction between the primary board and the secondary board is distinguished and identified by the model recognition module.

[0077] The PLC and the edge computing box adopt the Modbus-RTU protocol, and communicate through the 485 interface, with the edge computing box (T506) as the host and the PLC as the slave. The address of the holding register 40001 is the status code, and a total of 5 types of status codes are designed, including 0 for successful data reading, 1 for writing statistical data, 2 for camera abnormality, 3 for other abnormalities, 4 for model initialization, and 5 for system preparation; 40002-40005 are the number of four types of defects respectively, 40011-40012 are the dead knot and loose knot area settings respectively, and 400015 is the image saving setting. The four types of defects include dead knots, loose knots, bark and cracks; the model recognition module is loaded in the edge computing box.

[0078] exist Figure 2The system includes: a light shielding box 1, a light 2, a camera 3, a display 4, an edge computing box 5, a control box 6, a eucalyptus board 7, a photoelectric switch 8, and a conveyor belt 9. The light shielding box has dimensions (length, width, and height) of 130cm*80cm*85cm, and the conveyor belt 9 has a speed of 65 meters per minute. The camera 3 is 60cm above the eucalyptus board 7 below it, and is mounted on a retractable bracket with an adjustable range of 0-30cm. The eucalyptus board 7 measures 120cm*61cm (length and width). The installation position of the light 2 is divided into three parts. A light strip about 125cm long with a light intensity of 7500k is installed on the top, mainly to provide fill light for the eucalyptus board 7 directly below. The same light strips about 60cm long are installed on both sides of the light shielding box 1 to provide fill light for the sides of the eucalyptus board 7. In order to accurately capture the image of the eucalyptus board 7, a photoelectric switch 8 is installed in front of the light shielding box 1 to obtain the position information of the eucalyptus board 7 below the camera 3. A control box 6, measuring 30cm x 20cm x 20cm in length, width, and height, is designed to house the edge computing box (T506) 5. It can be connected to an external display via an HDMI interface, allowing users to easily identify the situation in real time. A user interface is also located on the outside of the control box 6, allowing users to set various information as needed.

[0079] Example 2

[0080] like Figure 3 、 4-1 , 4-2, and 5.

[0081] A detection method based on the detection system described in Example 1, comprising image acquisition, image training and image recognition,

[0082] In the image acquisition:

[0083] The Bbox-cover method is used to operate the collected original image dataset, as shown in the flowchart. Figure 3 , perform label balancing on each collected data image, the steps are as follows:

[0084] Step 1: Sequentially select an image from the dataset as the operation object F;

[0085] Step 2: Determine the label category of the operation object F to see if the operation object F has four types of defects q, including: q0 represents a dead knot, q1 represents a slip knot, q2 represents bark, and q3 represents a crack;

[0086] If a certain defect category is missing, perform the above Copy-paste operation on the operation object F:

[0087] The operation object F is used as the main image I main , for the missing defect category q iCopy to the operation object F, where i∈(0, 1, 2, 3). When the missing defect category qi is copied to the operation object F, specifically:

[0088] First, traverse the label file to obtain the missing defect category label;

[0089] Secondly, the defect image corresponding to the missing defect category label is copied and pasted to a random position of the operation object F. At the same time, the position information of the random position is updated to the defect category label. This is done until the operation object F meets the label determination condition and then enters Step 3.

[0090] Step 3: Count the defect labels and calculate the mean number of defect categories In combining each defect qi with Perform absolute value calculation to obtain q i- or q i+ ,q i- Represents the defect category that needs to be reduced, q i+ Represents the defect categories that need to be added among the four types of defects;

[0091] Get q i+ Defect image Bbox1 coordinate point (x min ,y min , x max ,y max ) to crop and cover q i- The coordinate point (x′ min , y′ min , x′ max , y′ max ) until the number a is met, where the Bbox coordinate points represent the upper left and lower right corner coordinate points of the target box after normalization, and β is the proportional coefficient:

[0092]

[0093]

[0094] The value of β is 0-1;

[0095] Step 4: Complete the above steps from Step 1 to Step 3 for each image in the entire dataset to achieve a balance in the defect categories of the entire dataset;

[0096] Image acquisition is combined with the specific processing steps of the system described in Example 1:

[0097] First, the light-shielding box 1 is placed above the conveyor belt 9 and connected to the edge computing box (T506) 5 via the USB cable of the camera 3. The photoelectric switch 8 is connected to the camera's six-core aviation head via a voltage follower. After configuration is complete, the device is powered on. The user enters the desired number of defects and the size of the defect area, and observes the change in the status bit on the display of the control box 6. If it changes to 6, the eucalyptus veneer 7 can be placed above the conveyor belt 9. This will drive the eucalyptus veneer 7 forward. When the eucalyptus veneer 7 reaches the position of the photoelectric switch 8, the photoelectric switch 8 will send an external trigger signal to the camera 3, capturing the original image. After the image is obtained, the original image is first saved locally, and the eucalyptus veneer 7 is placed again. Ultimately, approximately 2100 original images are collected. The collected images are first cleaned to remove duplicate images and images without eucalyptus veneers. Then, the Labimg tool is used to mark various defects, where 0 represents a dead knot, 1 represents a loose knot, 2 represents bark, and 3 represents a crack. The marks are marked in YOLO format.

[0098] In the image training:

[0099] The defects of dead knots, loose knots, cracks and bark in eucalyptus veneer also have different channel dimension information in the image. Therefore, the attention mechanism module SE-Net (Sequeeze and Excitation Net) is added to the YOLOv5 structure. Its structure is as follows Figure 4-1 As shown, to obtain the feature layer of the input image, the weight of each channel, the specific steps are as follows:

[0100] First, the global average pooling is used to output 1×1×C data:

[0101]

[0102] The formula (IV) is also called the Fsq (Squeeze) operation, which is the compression feature map operation in Chinese, where H represents the image height, W represents the image width, and the Fsq operation is performed on C feature maps of size H*W. uc represents the two-dimensional matrix of the channel information of each feature map, and the subscript c represents the number of channels of each feature map. Then comes the Fex (excitation) operation, which is the extraction feature map operation in Chinese. The formula is as shown in (V), where w1 is multiplied by the operation result of each feature map obtained by formula (IV) to obtain (1×1×C) / r, which is the first fully connected layer; the output dimension remains unchanged after passing through Relu; then it is multiplied by w2, which is the second fully connected layer, with an output dimension of 1×1×C, and δ is the Re1u function. This operation passes through two fully connected layers, and finally the data is normalized by the Sigmod function to obtain the result s:

[0103] s=F ex(z, w) = σ(w2δ(w1z)) (V)

[0104] In formula (V), z is the operation result of each feature map obtained by formula (IV); w is the secondary full connection operation, w1 represents the first operation, and w2 represents the second operation; σ is the sigmoid operation;

[0105] Finally, through F scale The operation, which is a recalibration operation in Chinese, multiplies the results of the Fsq operation and the Fex operation:

[0106] F scal e(u c , s c )=s c ·u c (VI)

[0107] In formula (VI), u c is a two-dimensional matrix of each feature map channel information; sc is the weight; through F scale To enhance important channel information, this technical feature uses SENet to obtain more features about the channel dimension of eucalyptus defects, ultimately forming a recognition model.

[0108] In the specific implementation, the model training is performed through the images and annotation files collected above. First, the YOLOv5 model structure yaml file is set, which includes the C3 module, SSP and other modules of the Backbone part; the Neck part includes the added four SE modules (a, b, c, d parts); the AGM aggregation module is added to the Head part to connect three detection heads of different scales, namely 76*76, 38*38, and 19*19.

[0109] The input raw eucalyptus veneer image undergoes preprocessing and is uniformly resized to 640*640. It then passes through three C3 blocks to output a 128-dimensional vector, which then passes through several C3 blocks to produce a 768-dimensional vector. This is followed by a 3×3 convolution to output a 1024-dimensional vector. This vector is then passed through the SPP block and fed into the Neck component. After a series of convolutions and upsampling operations, the SE block enhances the channel dimension. The image is then fed into the AGM block, where the input feature map is fused with pixel and channel information. Finally, a 1×1 convolution is performed and the image is fed into three detection heads of different sizes. Training is performed for 300 epochs with a batch size of 32. The initial learning rate is 0.01, the cyclic learning rate is 0.2, and the learning rate momentum is 0.937. SGD with momentum is used as the optimizer, and hyperparameter evolution is also implemented. The training environment is Ubuntu, based on the PyTorch framework, and accelerated by an NVIDIA Tesla A100 40GB graphics card with CUDA version 11.4. After many experiments, this embodiment also compares the present invention with other existing recognition methods, such as Figure 8 As shown, it is found that the improved YOL0v5 model is superior to other methods in terms of training time and mAP indicators, so the recognition model shown in the present invention is selected for deployment.

[0110] After training, the optimal model weights are obtained and deployed in the edge computing box (T506) 5. The process steps are as follows:

[0111] 1. First convert the above training weights into ONNX format.

[0112] 2. Lightweight the model and compress the model using pruning.

[0113] 3. Package the model into an SDK.

[0114] 4. Code encryption is performed by compiling it into an executable file and verifying and binding it to the hardware address. The T506 is configured with 48 Tensor Cores, providing 6.8TFlops of FP16 computing power, an HDIM interface, and support for 485 communication. We also implemented TensorRT to accelerate the model inference process.

[0115] The model in the edge computing box 5 is used for reasoning to obtain the recognition result graph, and the Flask framework is used to display the recognition result on the front-end page, and finally displayed on the display screen 4.

[0116] In the image recognition:

[0117] After using the recognition model to identify the eucalyptus defect image information, the recognition results are as follows: Figure 6 , use the following area formula to get the area value of each type of defect, filter out the total number of defects that meet the range according to the threshold set by the user, and complete the sorting task based on the total number. Calculate P(x p ,y p ) and Q(x q ,y q ) The distance between two pixels D(P, Q):

[0118]

[0119] Then, using the actual measured length and width of the eucalyptus, the actual distance of each pixel is obtained through formula (VIII):

[0120]

[0121] In formula (VIII), d w is the actual width of the eucalyptus veneer; d l is the length of the eucalyptus veneer; D w is the distance between the pixels of the eucalyptus veneer width in the image; D l is the distance in pixels of the length of the eucalyptus board in the image; the image size is 2048*2048px, and it is calculated that 1 pixel is approximately 0.70mm. The defect image is identified by the recognition model, the ROI area is extracted, and the circumscribed rectangular frame of the defect is drawn using image processing. Finally, the area of ​​the rectangular frame is calculated to obtain the area of ​​the defect image.

[0122] Example 3

[0123] As in the detection method described in Example 2, before using the Bbox-cover method, the copy-paste strategy is used to perform the following operations on the defect types such as slip knots, bark, and cracks, which are less common. The steps are as follows:

[0124] Through screening, images with slip knots, bark, and cracks are selected as source images I src , used to extract the defect image and paste it into the main image I main Random position in the source image I src and the main image I main Perform random horizontal inversion respectively, or perform random horizontal inversion on the source image I src Perform random scale scaling; finally, use formula (III) for synthesis:

[0125] I=I src ×α+I main ×(1-α) (III)

[0126] In formula (III), α is the mask, that is, the pixel area of ​​the defective part in the source image; I represents the new image after copy-paste.

[0127] Example 4

[0128] As described in the detection method of Example 2, the image training also includes an aggregation module AGM, such as Figure 4-2 As shown in the figure, the channel dimension information of the image is effectively combined with the pixel information, the input image channel dimension is maximized pooled, the pixel dimension information is averaged pooled, and finally spliced ​​and output to the detection head after completing the 1×1 convolution. The experiment found that with a small increase in the amount of calculation, the model map index increased by about 0.10. The overall modified network structure of YOLOv5 is shown in the figure. Figure 5 .

[0129] In summary, the present invention is used to identify eucalyptus veneers in actual scenarios:

[0130] The raw image data collected revealed an imbalance in the number of samples for each defect type within eucalyptus veneer: dead knots, slipknots, bark, and cracks. Approximately 2,100 images were initially collected, each measuring 2048px by 2048px. During the labeling process, we discovered a significant imbalance in the defect types within a single eucalyptus veneer, with dead knots accounting for approximately 80% and slipknots, bark, and cracks making up a smaller proportion. This dataset can significantly bias the trained model's weights toward dead knots, resulting in a decrease in the accuracy of recognition for other defect types. Therefore, we employed a custom-designed Bbox-cover strategy to enhance and cover images with fewer defects, ensuring a balanced model's recognition of all defect types.

[0131] Existing defect recognition methods often ignore too much channel dimension information for small target defects, resulting in unsatisfactory recognition results. Image segmentation methods such as Mask R-CNN can achieve high recognition accuracy, but it is difficult to meet the recognition speed requirements of actual scenarios. In terms of accuracy and speed of eucalyptus defect recognition, we use the single-stage target detection algorithm YOLOv5, and at the same time improve its model structure and add SENet (channel attention mechanism). Through this structure, the present invention can extract more information about the channel dimensions of various defects in eucalyptus veneers, and at the same time fuse the pixel information of the defect with the channel information based on the aggregation module. Figure 8 As shown in the figure, compared with other methods, the improved method can effectively improve the recognition accuracy of eucalyptus board defects.

[0132] In addition, in practice, the sizes of dead knots and loose knots vary, and different users have different requirements for the size of identified defects. In order to meet user needs, the present invention also designs a method for calculating the area size of various defects in eucalyptus boards and a user interaction page. Users can set the threshold range for the number and size of sorted defects according to their own needs.

[0133] The present invention uses the above detection equipment and detection method, based on high-speed board collection mechanical equipment, to design a set of eucalyptus veneer detection and sorting integrated equipment, and compared with other detection methods, Figure 9 The present invention can replace manual inspection and improve efficiency at the same time. The inspection and sorting efficiency can reach 50 sheets per minute, and the grading accuracy can reach about 90%.

Claims

1. A method for detecting defects in eucalyptus veneer images, characterized in that: The method is implemented based on a eucalyptus veneer defect image detection system. The system includes a high-speed board collector, an image acquisition module, a model training module, and a model recognition module. The image acquisition module uses the Bbox-cover method to operate on the collected original image dataset. The model training module adds the attention mechanism module SE-Net to the YOLOv5 structure to obtain the feature layer of the input image and the weight of each channel. The image acquisition module, model training module and model recognition module are installed in the high-speed board collecting device to collect, process, identify and detect the eucalyptus veneer entering the high-speed board collecting device; The eucalyptus veneer defect image detection system further includes: a sorting device, wherein the sorting device controls the forward and reverse rotation of a motor through a PLC to thereby control the direction of a grading baffle, wherein the PLC controls the forward and reverse rotation of the motor based on the eucalyptus veneer defect image detection results, wherein the detection results include first-level boards and second-level boards; If it is detected as a first-level board, the motor is controlled to rotate forward, and the eucalyptus veneer falls into the first-level board bin; If it is detected as a secondary board, the motor is controlled to reverse and the eucalyptus veneer falls into the secondary board bin; The PLC and the edge computing box use the Modbus-RTU protocol to communicate with each other through the 485 interface, with the edge computing box acting as the host and the PLC as the slave. The address of the holding register 40001 is the status code; 40002-40005 are respectively the number of four types of defects, 40011-40012 are respectively the area settings of dead knots and slip knots, and 400015 is the image saving setting. The four types of defects include dead knots, slip knots, bark and cracks. The model recognition module is loaded in the edge computing box. The eucalyptus veneer defect image detection method includes image acquisition, image training and image recognition: In the image acquisition: Use the Bbox-cover method to operate on the collected original image dataset. The steps are as follows: Step 1: Sequentially select an image from the dataset as the operation object F; Step 2: Determine the label category of the operation object F to see if the operation object F has four types of defects q. The defects q include: Indicates a deadlock, Expressing Slipknot, Represents bark, Expression cracks; If a certain defect category is missing, perform the following Copy-Paste operation on the operation object F: The operation object F is used as the main image , for the missing defect categories Copy to the operation object F, where , go to Step 3; Step 3: Count the defect labels and calculate the mean number of defect categories , in each defect and Perform absolute value calculation and get or , Represents the defect categories that need to be reduced. Represents the defect categories that need to be added among the four types of defects; Get Defect image Bbox1 coordinate point (x min ,y min ,x max ,y max ) to crop and cover The coordinate point (x' min ,y' min ,x' max ,y' max ), until the quantity is met So far, the Bbox coordinate points represent the upper left and lower right coordinate points of the target box after normalization. is the proportionality factor: （I） (II) The value of β is 0-1; Step 4: Complete the above steps from Step 1 to Step 3 for each image in the entire dataset; In the image training: The attention mechanism module SE-Net is added to the YOLOv5 structure to obtain the feature layer of the input image and the weight of each channel. The specific steps are as follows: First, the global average pooling is used to output 1×1×C data: (IV) The formula (IV) is also called Fsq operation, which means compressed feature map operation in Chinese, where H Represents the image height, W Represents the image width, and performs Fsq operation on C feature maps of size H*W. A two-dimensional matrix representing the channel information of each feature map, where the subscript c represents the number of channels of each feature map; Then comes the Fex operation, which is the feature map extraction operation in Chinese. The formula is as follows (V), where w 1 and the result of each feature map obtained by formula (IV) are multiplied to obtain (1×1×C) / r, which is the first fully connected layer; after passing through Relu, the output dimension remains unchanged; then w 2 multiplication, this is the second fully connected layer, the output dimension is 1×1×C, is the Relu function, and finally the data is normalized by the Sigmod function to obtain the result s : (V) In formula (V), z Obtain the operation result of each feature map for formula (IV); w is a secondary full connection operation, w 1 represents the first operation, w2 represents the second operation; ; Finally, through F scale The operation, which is a recalibration operation in Chinese, multiplies the results of the Fsq operation and the Fex operation: (WE) In formula (VI), u c A two-dimensional matrix of channel information for each feature map; is the weight; finally, a recognition model is formed; The image training also includes an aggregation module AGM, which performs maximum pooling on the input image channel dimension and average pooling on the pixel dimension information, and finally performs splicing and 1×1 convolution before outputting to the detection head; In the image recognition: After the eucalyptus defect image information is identified by the recognition model shown, the P( , ) and Q( , ) The distance between the two pixels : VII) Then, using the actual measured length and width of the eucalyptus tree, the actual distance of each pixel is obtained through formula (VIII) : (VIII) In formula (VIII), d w is the actual width of the eucalyptus veneer; d l is the length of the eucalyptus veneer; D w is the distance between the pixels of the eucalyptus veneer width in the image; D l is the distance in pixels of the length of the eucalyptus board single board in the image; the defect image is identified by the recognition model, the ROI area is extracted, the circumscribed rectangular frame of the defect is drawn using image processing, and finally the area of ​​the rectangular frame is calculated to obtain the defect image area.

2. The detection method according to claim 1, characterized in that For missing defect categories When copying to the operation object F, specifically: First, traverse the label file to obtain the missing defect category label; Secondly, the defect image corresponding to the missing defect category label is copied and pasted to a random position of the operation object F. At the same time, the position information of the random position is updated to the defect category label; until the operation object F meets the label judgment condition, enter Step 3.

3. The detection method according to claim 1, characterized in that Before using the Bbox-cover method, the following operations are performed on the defect categories of slipknot, bark, and crack using the copy-paste strategy. The steps are as follows: By filtering, select images with slip knots, bark, and cracks as source images I src ， Used to extract defect images and paste them into the main image Random position in the source image I src and the main image Perform random horizontal inversion or reversal of the source image I src Perform random scaling; Finally, the synthesis was performed using formula (III): (III) In formula (III), The mask is the pixel area of ​​the defective part in the source image; I represents the new image after copy-paste.

Images