Wood surface defect detection system, detection method and training method

Abstract

The present application relates to the technical field of wood defect detection, and particularly relates to a wood surface defect detection system, a detection method and a training method.A wood surface defect detection system is used for identifying wood surface defects on a wood picture.The detection system comprises a main module, a neck module and a feature head module.The main module comprises an input end and an output end.The neck module comprises a semantic fusion module, a feature pyramid network module and an adaptive structural feature fusion module.The output end of the main module is connected with the input end of the semantic fusion module and the input end of the feature pyramid network module respectively.The output end of the semantic fusion module is connected with the input end of the feature pyramid network module.The output end of the feature pyramid network module is connected with the input end of the adaptive structural feature fusion module.The output end of the adaptive structural feature fusion module is connected with the feature head module.The problem of slow detection speed of the existing detection model is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wood defect detection, and in particular to a wood surface defect detection system, a detection method and a training method. BACKGROUND

[0002] Wood is a natural renewable resource derived from trees and plays an important role in life and production. However, during the production process, defects such as knotting, cracking, discoloration, rotting and insect damage may occur on the surface of wood. These defects destroy the integrity and stability of the wood and affect production efficiency. As one of the quality standards for determining wood, accurate identification of the defects on the surface of wood is beneficial to production. The manual detection method has low detection efficiency, low precision and high cost due to high labor cost, different personal detection standards, high work intensity and poor working environment, thereby affecting the determination of the quality of wood. The progress of technology has led to the emergence of other methods for identifying wood defects, such as nuclear magnetic resonance detection, ultrasonic waves, infrared rays and lasers. However, the detection equipment used in the above methods is usually expensive, and the detection speed is slow when the wood is detected based on the above detection equipment, which is not suitable for widespread promotion. SUMMARY

[0003] In order to solve the problem of slow detection speed of the existing detection model, the present application provides a wood surface defect detection system.

[0004] To solve the above technical problems, the present application provides the following technical solution: a wood surface defect detection system for identifying wood surface defects on a wood picture, the detection system comprising a main module, a neck module and a feature head module;

[0005] The main module comprises an input end and an output end, the neck module comprises a semantic fusion module, a feature pyramid network module and an adaptive structural feature fusion module, the output end of the main module is connected to the input end of the semantic fusion module and the input end of the feature pyramid network module respectively, the output end of the semantic fusion module is connected to the input end of the feature pyramid network module, the output end of the feature pyramid network module is connected to the input end of the adaptive structural feature fusion module, and the output end of the adaptive structural feature fusion module is connected to the feature head module;

[0006] The wood picture obtains initial feature maps of different sizes based on the main module, the initial feature maps obtain optimized feature maps based on the semantic fusion module, part of the initial feature maps and the optimized feature maps obtain initial fusion feature maps through the feature pyramid network module, the initial fusion feature maps obtain optimized fusion feature maps after being input into the adaptive structural feature fusion module, and the optimized fusion feature maps obtain a defect map through the feature head module to identify the wood surface defects.

[0007] Preferably, the stem module comprises a composite module, a first feature aggregation module, a second feature aggregation module and a third feature aggregation module connected in sequence, the composite module comprises two base modules and an efficient layer aggregation network module connected in sequence, an input end of one base module is used for receiving the wood picture, an output end is connected with an input end of another base module, an output end of the another base module is connected with an input end of the efficient layer aggregation network module, an output end of the efficient layer aggregation network module is connected with an input end of the first feature aggregation module, output ends of the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are respectively connected with input ends of the neck module.

[0008] Preferably, the first feature aggregation module is used for outputting a first initial feature map, and output ends of the first feature aggregation module are respectively connected with input ends of the second feature aggregation module, the semantic fusion module and the feature pyramid network module;

[0009] The second feature aggregation module is used for outputting a second initial feature map, and output ends of the second feature aggregation module are respectively connected with input ends of the third feature aggregation module, the semantic fusion module and the feature pyramid network module;

[0010] The third feature aggregation module is used for outputting a third initial feature map, and an output end of the third feature aggregation module is connected with an input end of the semantic fusion module, the third initial feature map is converted into an optimized feature map based on the semantic fusion module, and an output end of the semantic fusion module is connected with an input end of the feature pyramid network module.

[0011] Preferably, sizes of the first initial feature map, the second initial feature map and the third initial feature map are different.

[0012] Preferably, the base module is composed of a convolution layer, a batch normalization layer and a basic module, and the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are each composed of an efficient layer aggregation network module and a maximum value pooling layer.

[0013] Preferably, the feature pyramid network module is used for converting the first initial feature map into a first initial fusion feature map, or converting the second initial feature map into a second initial fusion feature map, or converting the optimized feature map into a third initial fusion feature map.

[0014] An output end of the feature pyramid network module is connected with an input end of the adaptive structure feature fusion module, so as to convert the first initial fusion feature map into a first optimized fusion feature map, or convert the second initial fusion feature map into a second optimized fusion feature map, or convert the third initial fusion feature map into a third optimized fusion feature map.

[0015] To solve the above technical problems, the present application provides another technical solution as follows: a wood surface defect detection method applied to the wood surface defect detection system, the detection method comprising:

[0016] A wood picture and a backbone module are provided, and the wood picture is used to obtain initial feature maps of different sizes based on the backbone module;

[0017] A semantic fusion module is provided, and the initial feature map of the smallest size is selected and the optimized feature map is obtained by using the semantic fusion module;

[0018] A feature pyramid network module is provided, and the initial feature maps that are not selected and the optimized feature map are input into the feature pyramid network module to obtain initial fusion feature maps of multiple sizes;

[0019] An adaptive structure feature fusion module is provided, and all the initial fusion feature maps are input into the adaptive structure feature fusion module to obtain optimized fusion feature maps of multiple sizes;

[0020] A prediction head module is provided, and the optimized fusion feature map is used to obtain a defect map based on the prediction head module;

[0021] A defect result on the defect map is obtained and output.

[0022] Preferably, when all the initial fusion feature maps are input into the adaptive structure feature fusion module, the optimized fusion feature map of each size corresponds to all the initial fusion feature maps.

[0023] Preferably, the defect result includes any one or more combinations of a live knot, a dead knot, resin, a crack, a crack knot, a knot missing, a color difference, decay, and a pith.

[0024] To solve the above technical problems, the present application provides another technical solution as follows: a training method of a wood surface defect detection system applied to the wood surface defect detection system, the training method comprising:

[0025] A sample picture is provided, and the sample picture is converted into data;

[0026] The sample picture is cleaned, and the cleaned sample picture is subjected to affine transformation data enhancement processing to obtain a first enhanced picture;

[0027] copying and pasting the first enhanced picture to obtain a second enhanced picture;

[0028] obtaining a default anchor frame of the second enhanced picture, re-calculating the default anchor frame to obtain a new anchor frame, and performing iterative training after replacing the default anchor frame with the new anchor frame;

[0029] setting a maximum number of training times, and determining whether the number of the current training is greater than a preset number;

[0030] If not, sequentially performing mosaic data enhancement processing and mixed data enhancement processing on the second enhanced picture to obtain a training picture, and inputting the training picture into the wood surface defect detection system to update the data in the detection system;

[0031] If yes, directly inputting the second enhanced picture into the wood surface defect detection system to update the data in the detection system.

[0032] Compared with the prior art, the wood surface defect detection system, the detection method and the training method provided by the present application have the following beneficial effects:

[0033] 1. The wood surface defect detection system provided by the embodiment of the present application is used for identifying wood surface defects on a wood picture, and the detection system comprises a main trunk module, a neck module and a feature head module.

[0034] The main trunk module comprises an input end and an output end, the neck module comprises a semantic fusion module, a feature pyramid network module and an adaptive structure feature fusion module, the output end of the main trunk module is connected with the input end of the semantic fusion module and the input end of the feature pyramid network module respectively, the output end of the semantic fusion module is connected with the input end of the feature pyramid network module, the output end of the feature pyramid network module is connected with the input end of the adaptive structure feature fusion module, and the output end of the adaptive structure feature fusion module is connected with the feature head module.

[0035] The wood picture obtains initial feature maps of different sizes based on the main trunk module, the initial feature maps obtain optimized feature maps based on the semantic fusion module, part of the initial feature maps and the optimized feature maps obtain initial fusion feature maps through the feature pyramid network module, the initial fusion feature maps obtain optimized fusion feature maps after being input into the adaptive structure feature fusion module, and the optimized fusion feature maps obtain defect maps through the feature head module to identify wood surface defects. It should be understood that the semantic fusion module in the present application enhances the semantic and detail information in the image by integrating the hierarchical feature maps generated in the main trunk module. The adaptive structure feature fusion module improves the detection accuracy of the structural defects often present in the image edges. The wood surface defect detection system provided by the embodiment can not only identify various defects and various defect combinations, but also improve the efficiency and accuracy of image recognition compared with the prior art.

[0036] 9. The wood surface defect detection method according to the embodiment of the present application has the same beneficial effects as the wood surface defect detection system described above, and thus repeated description is omitted.

[0037] 10. The wood surface defect detection system training method according to the embodiment of the present application can produce training pictures to update the data in the wood surface defect detection system, so as to provide the accuracy of the wood surface defect detection system for wood surface defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a structural schematic diagram of the wood surface defect detection system provided by the first embodiment of the present application.

[0039] Figure 2 is a structural schematic diagram of the base module of the wood surface defect detection system provided by the first embodiment of the present application.

[0040] Figure 3 is a structural schematic diagram of the feature aggregation module of the wood surface defect detection system provided by the first embodiment of the present application.

[0041] Figure 4 is a network structural schematic diagram of the PAN module in the prior art.

[0042] Figure 5 is a network structural schematic diagram of the neck module in the wood surface defect detection system in the present application.

[0043] Figure 6 is a flowchart of the wood surface defect detection method provided by the second embodiment of the present application

[0044] Figure 7 is a flowchart of the wood surface defect detection system training method provided by the third embodiment of the present application

[0045] BRIEF DESCRIPTION OF DRAWINGS

[0046] 10. Wood surface defect detection system

[0047] 1. Main module; 2. Neck module; 3. Feature head module

[0048] 11. Compound module; 12. First feature aggregation module; 13. Second feature aggregation module; 14. Third feature aggregation module; 21. Semantic fusion module; 22. Feature pyramid network module; 23. Self-adaptive structural feature fusion module

[0049] 111. Base module; 112. Efficient layer aggregation network module; 121. Efficient layer aggregation network module; 122. Maximum pooling layer

[0050] 1111, convolution layer; 1112, batch normalization layer; 1113, basic module. DETAILED DESCRIPTION

[0051] For the purposes of the present application, the technical solutions and advantages will be clearer. The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are merely intended to explain the present application and not to limit the present application.

[0052] In the embodiments provided by the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that the determination of B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0053] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0054] In various embodiments of the present application, it should be understood that the size of the serial number of the above processes does not mean the inevitable sequence of execution, and the execution sequence of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0055] In the flowcharts and block diagrams of the drawings of the present application, the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments of the present application are illustrated. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks can also occur in a different order from that indicated in the drawings. For example, two blocks indicated in succession can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should be particularly noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0056] Wood is a natural renewable resource derived from trees, which plays an important role in life and production. However, during the production process, defects such as knotting, cracking, discoloration, rotting and insect damage may occur on the surface of wood. These defects destroy the integrity and stability of wood, affecting production efficiency. The defects on the surface of wood, as one of the criteria for determining the quality of wood, are accurately identified to facilitate production. Artificial detection methods have low efficiency, low precision and high cost due to high labor costs, different personal detection standards, high work intensity, poor working environment and other factors, which further affect the determination of wood quality. The progress of technology has led to the emergence of other methods for identifying wood defects, such as nuclear magnetic resonance detection, ultrasonic waves, infrared, laser, etc. However, the detection equipment used in the above methods usually has high cost, and there is a problem of slow detection speed when detecting wood based on the above detection equipment, which is not suitable for widespread promotion.

[0057] The present application is improved based on the existing image recognition network. The existing image recognition network mainly includes three structures: backbone, neck and head. Among them:

[0058] The backbone is the core part of the image recognition network, responsible for extracting basic feature information from the input image. It is usually composed of a deep convolutional neural network (CNN), which can gradually extract the features of the image from low to high levels.

[0059] The neck connects the backbone and the head, and is usually used for further processing and optimization of the fused feature map. Its goal is to fuse and enhance the multi-level features extracted from the backbone, so that the feature map is more effective in subsequent prediction.

[0060] The head is responsible for target detection, classification or regression tasks according to the output features of the neck, and finally generates prediction results. It converts the feature map into the final detection frame, class and confidence, etc.

[0061] Generally, a module for processing and extracting features of a graph and a module for extracting features are usually arranged in the backbone module. Exemplarily, a spatial pyramid pooling fast (SPPF) module is usually arranged in an existing image recognition network model. When an image is input into the backbone network, the SPPF performs multi-scale pooling on the input feature map, and splices the pooled results to obtain a multi-scale feature representation. The obtained feature representation can be directly input into the neck module, and the feature map is optimized and fused by a path aggregation network (PAN) module in the neck module, which can improve the robustness and performance of the network. Generally, after being fused by the PAN, the network can enter a head network (Head) to output a detection result.

[0062] Although the image recognition network of the prior art can recognize images, the types of wood surface defects are various. Exemplarily, a wood surface includes nine defects, and the specific defects include a live knot, a dead knot, resin, a crack, a crack knot, a knot loss, a color difference, decay, and a pith. That is, when a complex wood surface needs to be recognized, the existing image recognition network can only recognize a single defect, and a large network model parameter needs to be used, which is costly. In addition, the existing image recognition method also has the problem of inaccurate recognition.

[0063] Referring to Figure 1 The first embodiment of the present application provides a wood surface defect detection system 10 for recognizing wood surface defects on a wood picture. The detection system includes a backbone module 1, a neck module 2, and a feature head module 3. The backbone module 1 includes an input end and an output end. The neck module 2 includes a semantic fusion module 21, a feature pyramid network module 22, and an adaptive structural feature fusion module 23. The output end of the backbone module 1 is connected to the input end of the semantic fusion module 21 and the input end of the feature pyramid network module 22, respectively. The output end of the semantic fusion module 21 is connected to the input end of the feature pyramid network module 22. The output end of the feature pyramid network module 22 is connected to the input end of the adaptive structural feature fusion module 23. The output end of the adaptive structural feature fusion module 23 is connected to the feature head module 3.

[0064] It needs to be understood that the source of the wood picture is to take a picture of the real wood, and the picture taken can be a single wood or multiple woods. There can be many defects on the wood, such as knots, dead knots or cracks. The prior art is low in both recognition accuracy and recognition efficiency. In the embodiment, a new type of wood surface defect detection system 10 is provided. The main module 1 has an input end, and the wood picture can be input into the main module 1 through the input end, and different size initial feature maps can be obtained based on the main module 1. It should be noted that the size and content of the initial feature map have changed compared with the wood picture. After the initial feature map is fused based on the semantics and detail infusion module 21, the optimized feature map can be obtained. It should be noted that the semantics and detail infusion module 21 (Semantics and Detail Infusion, SDI) is part of the system. The hierarchical features generated by the network in the main module 1 can be divided into low-level features and high-level features. With the increase of network depth, the image features gradually become abstract. The low-level features retain more image details, but lack sufficient semantic information and may contain unwanted noise. High-level features contain more semantic information, but due to the significant reduction in resolution, they may result in the loss of detailed information such as target location information. The SDI module enhances the semantic and detail information in the image by integrating the hierarchical feature maps generated in the main module 1. That is, the semantics and details in the optimized feature map are enhanced compared with the initial feature map.

[0065] Further, part of the initial feature map and the optimized feature map obtain the initial fusion feature map through the feature pyramid network module 22 (Feature Pyramid Network, FPN). It should be noted that the FPN is proposed for the multi-scale problem in target detection, and the purpose is to improve the detection performance of small objects through simple network connection. That is, when the initial feature map and the optimized feature map enter the pyramid network module, the detection ability of small objects in the initial fusion feature map obtained will be improved.

[0066] Further, the initial fusion feature map is input into the adaptive structure feature fusion module 23 (Adaptive Structure Feature Fusion, referred to as ASFF) to obtain an optimized fusion feature map. The adaptive structure feature fusion module 23 uses Hadamard product operation to dynamically select and fuse multi-layer feature maps, so that the network can capture multi-scale information of the target. Hadamard product, also known as element-wise product, dot product or Schur product, is a special operation between two matrices. It produces a new matrix, and each element in the new matrix is the product of the corresponding elements in the original two matrices. Specifically, for two matrices A and B of the same shape, their Hadamard product C is defined as C = A x B, where Cij = Aij x Bij, where i and j represent the rows and columns of the matrix, respectively. In the existing network structure, the model's path aggregation network (the Path Aggregation Network, referred to as PAN) is often used. PAN is a deep learning network structure mainly used for image classification and target detection tasks. It can effectively improve the performance and accuracy of the model through multi-scale feature fusion and path aggregation. In PAN, by introducing top-down and bottom-up path aggregation, different levels of feature maps can be effectively utilized. Exemplarily, the top-down path aggregation module (PA): starting from the lower resolution feature map, the information is transmitted layer by layer upwards through the upsampling operation. At each level, the low-resolution feature map is connected horizontally and added point by point with the high-resolution feature map to integrate information of different scales. The bottom-up path aggregation module (PAM): starting from the high-resolution feature map, the information is transmitted layer by layer downwards through the downsampling operation. Similarly, at each level, the high-resolution feature map is connected horizontally and added point by point with the low-resolution feature map.

[0067] It should be noted that in the feature fusion part of the present embodiment, the top-down path in the PAN is replaced by the adaptive structure feature fusion module 23. When the existing PAN module fuses network information, the model in the system becomes complex. However, the adaptive structure feature fusion module 23 can make the network capture multi-scale information of the target. In addition, the use of the adaptive structure feature fusion module 23 can improve the mAP by 0.3% without reducing the overall mAP, making the model lightweight and reducing the number of model parameters by 10%. It should be noted that mAP (mean Average Precision) is an important indicator for evaluating the performance of a model. In addition, the adaptive structure feature fusion module 23 also improves the detection accuracy of the missing joints often appearing at the edges of the image, with an improvement effect of about 5%. That is, compared with the initial fusion feature map, the detection accuracy of the optimized fusion feature map is further improved.

[0068] Further, the optimized fusion feature map obtains the defect map through the feature head module 3, and the defect map is compared with the initial wood picture, and the network has improved the identification ability for wood defects. In addition, the experimental results on the data set show that the mAP of the improved network on the test set is increased by 4.5% than the original network. The improved network improves the detection precision without significantly increasing the model size and inference speed. The improved network improves the feature extraction ability of the image, can identify various wood surface defects, and the method can meet the accuracy and real-time demand in actual production. The differences between various wood surface defects include inter-class differences, intra-class differences and scale differences. Among the differences, the inter-class difference is small, which will affect the model classification. Exemplarily, the size difference of knots is extremely large, the length of cracks is not uniform, and the scale difference greatly affects the detection precision of different sizes of the same defect. By using the SDI module to integrate the network of the backbone module 1 to generate different size pictures and using the ASFF module to fuse useful information, the multi-size feature maps are fully communicated with useful information, and the defect size difference problem is alleviated. The feature extraction ability of dense targets and mixed targets is also improved.

[0069] Optionally, the wood defect identification in the embodiment includes any one or a combination of the following: live knot, dead knot, resin, crack, crack knot, knot loss, color difference, decay, and pith. It should be understood that, compared with the existing identification system, the wood surface defect detection system 10 provided in the embodiment can not only identify multiple defects and multiple defect combinations, but also improve the efficiency and accuracy of image recognition.

[0070] Further, please combine Figure 2 The backbone module 1 includes a composite module 11, a first feature aggregation module 12, a second feature aggregation module 13 and a third feature aggregation module 14 connected in sequence, the composite module 11 includes two basic modules 111 and an efficient layer aggregation network module 112 connected in sequence, the input end of one basic module 111 is used for receiving a wood picture, the output end is connected with the input end of the other basic module 111, the output end of the other basic module 111 is connected with the input end of the efficient layer aggregation network module 112, the output end of the efficient layer aggregation network module 112 is connected with the input end of the first feature aggregation module 12, and the output ends of the first feature aggregation module 12, the second feature aggregation module 13 and the third feature aggregation module 14 are connected with the input end of the neck module 2 respectively.

[0071] It should be noted that the backbone module 1 is the core part of the image recognition network, responsible for extracting basic feature information from the input image. The design of the backbone module 1 is particularly important. Among them, when the backbone picture is input from the input end of the composite module 11, the size of the wood picture will change. For example, if the size of the picture is "640x640" after passing through the first basic module 111, the size of the picture will change to "320x320", and after passing through the second basic module 111, the size of the picture will change to 1 / 4 of the initial wood picture, that is, the size will change to "160x160". Then the wood picture with a size of 1 / 4 will pass through the efficient layer aggregation network module 112 (Elan), which combines the outputs of multiple convolution layers to improve the efficiency and richness of feature extraction.

[0072] Further, in combination with Figure 3 , the 1 / 4 wood picture will pass through the efficient layer aggregation network module 112 and enter the first feature aggregation module 12. It should be noted that the first feature aggregation module 12, the second feature aggregation module 13 and the third feature aggregation module 14 are composed of an efficient layer aggregation network module 121 (Elan) and a maximum pooling layer 122 (Maxpool). The maximum pooling layer 122 is a common pooling operation in convolutional neural networks. It divides the input feature map into regions by sliding window, selects the maximum value in each region, and outputs the pooled feature map. The maximum pooling operation can effectively reduce the size of the feature map, reduce the computational complexity and memory occupation. In addition, the maximum pooling operation can also select the maximum value in the region, and the maximum pooling retains the significant features of the image and suppresses the noise.

[0073] Specifically, the sizes of the first initial feature map, the second initial feature map and the third initial feature map are different. The size of the 1 / 4 wood picture changes once after passing through each feature aggregation module. In addition, the output end of each feature aggregation module is the output channel of the initial feature map. It should be understood that when the 1 / 4 wood picture is input into the first feature aggregation module 12, the 1 / 4 wood picture will change into a first initial feature map with a size of 1 / 8. The first feature aggregation module 12 has three output ends, and the first feature aggregation module 12 is used to output the first initial feature map, wherein the first output end is connected with the input end of the second feature aggregation module 13, the second output end is connected with the input end of the feature pyramid network module 22, and the third output end is connected with the input end of the semantic fusion module 21. The first feature aggregation module 12 makes the first initial feature map with a size of 1 / 8 enter the second feature aggregation module 13, the feature pyramid network module 22 and the semantic fusion module 21 respectively.

[0074] Specifically, the second feature aggregation module 13 is configured to output the second initial feature map, and the output end of the second feature aggregation module 13 is connected with the input end of the third feature aggregation module 14, the semantic fusion module 21 and the feature pyramid network module 22 respectively. It should be understood that when the first initial feature map of 1 / 8 enters the second feature aggregation module 13, it will change into the second initial feature map of 1 / 16, and the second feature aggregation module 13 also has three output ends, of which the first output end is connected with the input end of the third feature aggregation module 14, the second output end is connected with the input end of the feature pyramid network module 22, and the third output end is connected with the input end of the semantic fusion module 21. The second feature aggregation module 13 makes the first initial feature map of 1 / 16 enter the third feature aggregation module 14, the feature pyramid network module 22 and the semantic fusion module 21 respectively.

[0075] Specifically, the third feature aggregation module 14 is configured to output the third initial feature map, and the output end of the third feature aggregation module 14 is connected with the input end of the feature pyramid network module 22, and the third initial feature map is converted into the optimized feature map based on the semantic fusion module 21, and the output end of the semantic fusion module 21 is connected with the input end of the feature pyramid network module 22. It should be understood that when the second initial feature map of 1 / 16 enters the third feature aggregation module 14, it will change into the third initial feature map of 1 / 32, and the third feature aggregation module 14 only has one output end. The third feature aggregation module 14 makes the third initial feature map of 1 / 32 enter the semantic fusion module 21. Further, the third initial feature map entering the semantic fusion module 21 will be processed by the semantic fusion module 21 to enhance the semantics and details in the picture and convert the third initial feature map into the optimized feature map, and the optimized feature map enters the feature pyramid network module 22 again.

[0076] It can be understood that after the processing of the main module 1 and the neck module 2 in the embodiment, the feature pyramid network module 22 can receive the first initial feature map, the second initial feature map and the optimized feature map. The sizes and contents of the first initial feature map, the second initial feature map and the optimized feature map are different. It should be understood that different sizes of pictures are input into the feature pyramid network module 22 for fusion, which can increase the richness of the pictures.

[0077] It should be noted that the traditional backbone module 1 usually enters the spatial pyramid pooling module (SPPF) after processing and aggregating the graph. After the backbone network extracts the feature maps of different levels, the SPPF fuses the information of different scales together through an efficient pooling method. The SPPF module achieves superior performance, but the SPPF module only processes the highest level features, and the feature fusion is not sufficient. In the present embodiment, the SPPF module is deleted from the inside of the backbone module 1, and a semantics and detail infusion module 21 (SDI) is added in the neck module 2. The SDI module can more effectively integrate the hierarchical features generated by the backbone module network. Specifically, the SDI module effectively integrates the shallow and deep features, and the fused feature maps help to alleviate the influence of scale difference on accuracy. The introduction of this module improves the overall detection of wood defects by 2.1%mAP, and significantly improves the detection accuracy for defects with large scale changes, such as knots, cracks and knot cracks, with detection accuracy improved by 5%, 5% and 4% respectively.

[0078] Specifically, the base module 111 (CBS) is composed of a convolutional layer 1111, a batch normalization layer 1112 (Batch Normalization), and a basic module 1113 (Sigmoid-weighted Linear Unit, SiLU for short). It should be understood that LReLU (Leaky Rectified Linear Unit) or ReLU (Rectified Linear Unit) is usually arranged in the existing base module 111, and LReLU and ReLU are a commonly used activation function and are widely used in neural networks. The activation function ReLU has high computational efficiency and enables the model to quickly converge during the learning process. When the input of the ReLU function is negative or zero, the gradient is zero, and the phenomenon of gradient disappearance is easy to occur. The activation function LReLU optimizes the defects of ReLU in the negative value region. When the network input is negative, it can also be back propagated, so LReLU can effectively prevent the phenomenon of gradient disappearance. However, the function curve of LReLU is discontinuous at point 0, and non-differentiability is its disadvantage. The embodiment selects to delete LReLU or ReLU, and selects to increase SiLU. The SiLU activation function is continuously differentiable, and its function graph is smoother than ReLU and LReLU. It avoids the non-differentiable problem of ReLU and has the same performance in the negative value region as LReLU. The SiLU activation function with non-monotonicity can better introduce a non-linear factor, which helps to improve the expression ability of the model and effectively improve the detection accuracy. It should be noted that in order to better apply to the field of wood surface defect detection, the modules in the system are redesigned, so that the wood surface defect detection system 10 can solve the problems that the traditional system cannot solve.

[0079] Exemplarily, the differences between various wood surface defects include inter-class differences, intra-class differences, and scale differences. Among the differences, the inter-class difference is small, which will affect the model classification, such as the shape similarity between the dead knot and the live knot in the wood surface defects, and the intra-class difference is large, which will affect the correct judgment of the defect type. In the embodiment, the SiLU in the base module 111 is innovatively increased, and the SiLU activation function in the base module 1113 effectively alleviates the intra-class difference and inter-class difference problem by introducing a non-linear factor.

[0080] Further, the feature pyramid network module 22 is configured to convert the first initial feature map into a first initial fusion feature map, or convert the second initial feature map into a second initial fusion feature map, or convert the optimized feature map into a third initial fusion feature map.

[0081] The output end of the feature pyramid network module 22 is connected with the input end of the adaptive structure feature fusion module 23, so as to convert the first initial fusion feature map into a first optimized fusion feature map, or convert the second initial fusion feature map into a second optimized fusion feature map, or convert the third initial fusion feature map into a third optimized fusion feature map.

[0082] It should be understood that the feature pyramid network module 22 improves the detection performance of small objects through simple network connection, so that the first initial feature map, the second first initial feature map and the optimized feature map are converted into the initial fusion feature map. Further, the key to improve the detection capability of the model is how to effectively integrate low-level and high-level features. The network of the traditional PAN module adds a bottom-up path. The increase of the path makes the network of the PAN module perform well in the traditional detection system. However, this also increases the complexity of the model feature fusion part in the system. In the embodiment, the network of the adaptive structure feature fusion module 23 filters the conflicting information in the space through the two steps of identity scaling and adaptive fusion, and reduces the fusion of useless information. The network of the adaptive structure feature fusion module 23 improves the inconsistency between different scale features in the feature pyramid network, and the number of network parameters is less than that of the network of the PAN module. In the adaptive structure feature fusion module 23, the feature maps from different layers are convolved with a 1x1 kernel size to obtain corresponding weight parameters. By multiplying and adding the weight parameters of the features from different layers, a new fusion feature can be obtained.

[0083] Please refer to Figure 4 , Figure 4 The network structure of the traditional PAN module is shown in the figure. As can be seen, when the initial fusion feature map is transported from the FPN module to the PAN module, the fusion feature map F6 is converted into the fusion feature map P6, the fusion feature map F7 is converted into the fusion feature map P7, and the fusion feature map F8 is converted into the fusion feature map P8. In the process of picture feature fusion, the fusion feature map P6 often refers to the features of the fusion feature map F8. For example, the fusion feature map P6 refers to the features of the fusion feature map F8, and the network flow path is: the fusion feature map F8 is first converted into the fusion feature map P8, the fusion feature map P8 is then fused into the fusion feature map P7, and the fusion feature map P7 is then fused into the fusion feature map P6. This fusion method is relatively indirect, that is, if the fusion feature map P6 wants to refer to the features of the fusion feature map F8, it must refer to the features of the fusion feature map F8 and the fusion feature map F7. This increases the complexity of the model feature fusion part and reduces the efficiency of the entire feature fusion.

[0084] Please refer to Figure 5 , Figure 5Fig. 2 is a schematic diagram of a network structure of the neck module 2 in the embodiment. The feature pyramid network module 22 (Feature Pyramid Network, FPN for short) and the adaptive structure feature fusion module 23 (Adaptive Structure Feature Fusion, ASFF for short) are two different network structures. By way of example, after the feature pyramid network module 22 is fused, the initial feature map B3 is converted into the initial fusion feature map F3, the initial feature map B4 is converted into the initial fusion feature map F4, and the optimized feature map S5 is converted into the initial fusion feature map F5. Further, when the initial fusion feature map F3, the initial fusion feature map F4, and the initial fusion feature map F5 are transmitted to the adaptive structure feature fusion module 23, the optimized fusion feature map A3 is fused by directly referring to the initial fusion feature map F3, the initial fusion feature map F4, and the initial fusion feature map F5, the optimized fusion feature map A4 is fused by directly referring to the initial fusion feature map F3, the initial fusion feature map F4, and the initial fusion feature map F5, and the optimized fusion feature map A5 is fused by directly referring to the initial fusion feature map F3, the initial fusion feature map F4, and the initial fusion feature map F5. That is, the biggest difference between the adaptive structure feature fusion module 23 and the traditional PAN module is that the adaptive structure feature fusion module 23 can adaptively select fusion, and the fusion manner is more direct and efficient. Specifically, the use of the adaptive structure feature fusion module 23 can improve the mAP by 0.3% without reducing the overall mAP, make the model lightweight, and reduce the parameter quantity of the model by 10%. In addition, the detection accuracy of the missing edge of the image can be improved by about 5%.

[0085] It should be noted that the sizes of the initial feature map, the optimized feature map, the initial fusion feature map, and the optimized fusion feature map have a corresponding relationship. By way of example, the network architecture of the initial feature map B3, the initial fusion feature map F3, and the optimized fusion feature map A3 is different, but the size of the picture is the same. Similarly, the network architecture of the initial feature map B4, the initial fusion feature map F4, and the optimized fusion feature map A4 is different, but the size of the picture is the same. Details are not described herein.

[0086] The second embodiment of the present application also provides a wood surface defect detection method applied to the wood surface defect detection system. The detection method comprises the following steps.

[0087] S1, providing a wood picture and a backbone module, and obtaining initial feature maps of different sizes based on the backbone module;

[0088] S2, providing a semantic fusion module, selecting the initial feature map with the smallest size, and obtaining an optimized feature map by using the semantic fusion module;

[0089] S3, a feature pyramid network module is provided, and the unselected initial feature map and the optimized feature map are input into the feature pyramid network module to obtain initial fusion feature maps of multiple sizes;

[0090] S4, an adaptive structural feature fusion module is provided, and all the initial fusion feature maps are input into the adaptive structural feature fusion module to obtain optimized fusion feature maps of multiple sizes;

[0091] S5, a prediction head module is provided, and the optimized fusion feature map obtains a defect map based on the prediction head module;

[0092] S6, a defect result on the defect map is obtained and output.

[0093] It can be understood that in the embodiment, the wood needs to be photographed first, and after the photographing is completed, the photographed wood picture can be input into the wood surface defect detection system. The wood picture can be input into the trunk module, and initial feature maps of different sizes can be obtained based on the trunk module. For example, if the size of the wood picture is “640x640”, there are three sizes of initial feature maps, and the sizes of the initial feature maps can be “80x80”, “40x40” and “20x20” respectively.

[0094] Further, the smallest feature map in the initial feature map is selected, and an optimized feature map is obtained by using a semantic fusion module. For example, the size of “20x20” in the above three initial feature maps is the smallest, so the initial feature map of “20x20” will be converted into an optimized feature map by the semantic fusion module. Further, the unselected initial feature map and the optimized feature map are input into the feature pyramid network module. That is, the initial feature maps of “80x80” and “40x40” and the optimized feature map of “20x20” will be input into the feature pyramid network module to be converted into initial fusion feature maps of multiple sizes. It should be noted that when the initial fusion feature map is converted, the size of the initial fusion feature map corresponds to the unconverted initial feature map or optimized feature map. It should be understood that the initial feature map of “80x80” will be converted into an initial fusion feature map of “80x80”, and the optimized feature map of “20x20” will be converted into an initial fusion feature map of “20x20”. It should be understood that the detection ability of small objects in the initial fusion feature map will be improved.

[0095] Further, all the initial fusion feature maps are input into the adaptive structural feature fusion module to obtain optimized fusion feature maps of multiple sizes. It should be understood that compared with the initial fusion feature map, the detection accuracy of the optimized fusion feature map is further improved.

[0096] It should be noted that when all the initial fusion feature maps are input into the adaptive structure feature fusion module, the optimized fusion feature map of each size corresponds to all the initial fusion feature maps. For example, the initial fusion feature map of "80x80" will be converted into an optimized fusion feature map of "80x80", and the initial fusion feature map of "20x20" will be converted into an optimized fusion feature map of "20x20".

[0097] Further, a prediction head module is provided, and the optimized fusion feature map obtains the defect map based on the prediction head module. Compared with the initial wood picture, the network will be difficult to identify and detect the wood defects on the wood picture, and when the wood picture is converted into the defect map, the network will quickly and accurately obtain the defect result on the defect map and output the defect result. Optionally, the defect result includes any one or a combination of the following: live knot, dead knot, resin, crack, crack knot, knot loss, color difference, decay, and pith.

[0098] It should be understood that the prior art detects few defect types or only detects a single type, or uses a large number of network model parameters, or uses data collected in an ideal environment for training, which does not conform to the actual situation of complex and diverse defect types and complex detection environment in industrial production. The data set used in the method is obtained in an actual industrial production environment, and a large amount of real data is obtained from the production line, which can detect live knots, dead knots, resins, cracks, crack knots, knot losses, color differences, decays, and piths. The data set obtained by using the data enhancement technology is sufficient in quantity, and the number of various defects in the data set is balanced. The use of the data enhancement technology avoids the increase in cost caused by the collection, cleaning, and labeling of image data in actual application, and at the same time, the technology can help the network better understand the differences in data and enhance the recognition ability of a small number of defects, improve the generalization ability of the model, and reduce the overfitting of the network to the training data. After using the data enhancement technology, the recognition ability of the network for defects is improved.

[0099] The third embodiment of the present application also provides a training method of a wood surface defect detection system, applied to the wood surface defect detection system, and the training method comprises the following steps:

[0100] A1, providing sample pictures, and cleaning the sample pictures;

[0101] A2, performing affine transformation data enhancement processing on the cleaned sample pictures to obtain first enhanced pictures;

[0102] A3, performing copy-paste data processing on the first enhanced pictures to obtain second enhanced pictures;

[0103] A4, obtaining a default anchor frame of the second enhanced picture, re-calculating the default anchor frame to obtain a new anchor frame, and replacing the default anchor frame with the new anchor frame to perform iterative training;

[0104] A5, setting a maximum number of iterations of the iterative training, and determining whether the number of the current training is greater than the preset number;

[0105] A6, if not, sequentially performing mosaic data enhancement processing and mixed data enhancement processing on the second enhanced picture to obtain a training picture, and inputting the training picture into the wood surface defect detection system to update the data in the detection system;

[0106] A7, if yes, directly inputting the second enhanced picture into the wood surface defect detection system to update the data in the detection system.

[0107] It can be understood that the traditional data training method usually performs data processing on the input sample picture, and then directly supplements the processed picture into the system. The problem faced by this method is that the sample picture may be different from the picture of the tree in the real world, that is, the sample picture is used for training the model, which does not fit the actual defect situation of the distribution of the tree in the real world.

[0108] In order to solve the technical problem, the sample picture is first cleaned in the embodiment. The method of cleaning the sample picture is specifically: correcting the image label in the public wood surface defect dataset, cropping the black edge of the dataset image, and converting the TXT label file into a VOC format XML file. It should be understood that when the network reads the wood surface defect in the picture, the format of the picture will have requirements, and converting the TXT into the XML file format is more suitable for the network to read and identify the defects on the picture in the system.

[0109] Further, the sample picture after cleaning is subjected to affine transformation data enhancement processing to obtain a first enhanced picture. It should be understood that the sample picture itself may have problems of unclear light and dark of the picture, and picture skew. The affine transformation data enhancement processing in the embodiment mainly adjusts the rotation of the internal graphics of the sample picture, the light and dark of the graphics, and whether to add noise in the graphics, that is, the main function of the data enhancement processing is to enrich the content of the sample picture. After the affine transformation data enhancement processing, the anti-interference and robustness of the picture are enhanced, so that the picture is more consistent with reality.

[0110] Further, the first enhanced picture is subjected to copy-paste data processing to obtain a second enhanced picture. It should be understood that the frequency of occurrence of defects on the wood picture is different, for example, some defects occur more frequently, and some defects occur less frequently. In the process of pattern recognition, the designer often expects to balance the phenomenon that the frequency of occurrence of these defects is different. When training the model, if the frequency of occurrence of defects is high, the training amount will be too much, and if the frequency of occurrence of defects is low, the training amount will be too little. After copy-paste data processing, the frequency of occurrence of different defects will be balanced so as to subsequently train different defects evenly.

[0111] Further, the new anchor box is reset, and the new anchor box is used to replace the default anchor box. It should be understood that the size of the sample picture is usually different from the picture size adapted by the network, so it is necessary to reset the anchor box of the picture. Exemplarily, the size of the sample picture is usually “2800*1440”, and the picture size adapted by the network is “640*640”. If the size of the sample picture of “2800*1440” is directly trained, the training speed of the system will be affected. It should be understood that after resetting the anchor box, the training speed will be greatly improved.

[0112] Specifically, the purpose of iterative training is to make the trained picture more consistent with the defect distribution of wood in the real world. Exemplarily, the iterative training can be to select the Adam (Adaptive Moment Estimation) optimizer, the initial learning rate is 0.001, and the cosine annealing method is used to update the learning rate, the training epoch is 120, and the batch size is 16. The training epoch refers to one complete cycle of forward propagation and backward propagation of the entire training data set on the neural network. Batch Size: Batch size refers to the number of samples processed by the model in one forward and backward propagation. The selection of batch size will affect the stability and speed of training. It should be noted that Adam is a high-efficiency optimization algorithm used for parameter updating in deep learning, which aims to adaptively adjust the learning rate of each parameter to improve the convergence speed and generalization ability of the model.

[0113] Further, it is judged whether the number of training is greater than the preset number. For example, it is assumed that the preset number is 70% of the maximum iteration number. That is, when 70% of the iteration number is completed, the Mosaic data augmentation processing and the Mixup data augmentation processing are closed after the training round. It should be understood that the Mosaic data augmentation processing is to splice four pictures after random cropping into a new picture as training data. The Mosaic data augmentation processing increases the diversity of data and enriches the background information of the picture. The Mixup data augmentation processing is to mix two random samples in proportion. The Mixup data augmentation processing will not appear unnatural after image mixing.

[0114] It should be understood that if the number of training is less than the preset number, the second enhanced picture needs to be subjected to the Mosaic data augmentation processing and the Mixup data augmentation processing to obtain the training picture, and then the training picture is input into the wood surface defect detection system to update the data in the detection system.

[0115] If the number of training is greater than the preset number, the second enhanced picture does not need to be subjected to the Mosaic data augmentation processing and the Mixup data augmentation processing, and can be directly input into the wood surface defect detection system to update the data in the detection system, which greatly improves the training efficiency.

[0116] It can be understood that the training method of the wood surface defect detection system provided by the embodiment can produce training pictures to update the data in the wood surface defect detection system, so as to provide the accuracy of the wood surface defect detection system for wood surface defect detection.

[0117] The wood surface defect detection system, the detection method and the training method disclosed in the above embodiment are described in detail. The principle and implementation mode of the present application are described by applying specific examples. The above embodiment is only used to help understand the method and core idea of the present application. For those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. The above description should not be understood as a limitation of the present application. Any modification, equivalent replacement and improvement within the principle of the present application should be included in the protection scope of the present application.

Claims

1. A wood surface defect detection system for identifying wood surface defects on a picture of wood, the system comprising: The detection system comprises a trunk module, a neck module and a feature head module; The trunk module comprises an input end and an output end, the neck module comprises a semantic fusion module, a feature pyramid network module and an adaptive structure feature fusion module, the output end of the trunk module is connected with the input end of the semantic fusion module and the input end of the feature pyramid network module respectively, the output end of the semantic fusion module is connected with the input end of the feature pyramid network module, the output end of the feature pyramid network module is connected with the input end of the adaptive structure feature fusion module, and the output end of the adaptive structure feature fusion module is connected with the feature head module. The wood picture obtains initial feature maps of different sizes based on the trunk module, the initial feature maps obtain optimized feature maps based on the semantic fusion module, part of the initial feature maps and the optimized feature maps obtain initial fusion feature maps through the feature pyramid network module, the initial fusion feature maps obtain optimized fusion feature maps after being input into the adaptive structure feature fusion module, and the optimized fusion feature maps obtain a defect map through the feature head module to identify wood surface defects; the trunk module comprises a composite module, a first feature aggregation module, a second feature aggregation module and a third feature aggregation module connected in sequence, the composite module comprises two basic modules and an efficient layer aggregation network module connected in sequence, the input end of one basic module is used to receive the wood picture, the output end is connected with the input end of the other basic module, the output end of the other basic module is connected with the input end of the efficient layer aggregation network module, and the output end of the efficient layer aggregation network module is connected with the input end of the first feature aggregation module, and the output ends of the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are connected with the input end of the neck module respectively.

2. The wood surface defect detection system of claim 1, wherein: The first feature aggregation module is used to output a first initial feature map, and the output end of the first feature aggregation module is connected with the input end of the second feature aggregation module, the semantic fusion module and the feature pyramid network module respectively; The second feature aggregation module is used to output a second initial feature map, and the output end of the second feature aggregation module is connected with the input end of the third feature aggregation module, the semantic fusion module and the feature pyramid network module respectively; The third feature aggregation module is used to output a third initial feature map, and the output end of the third feature aggregation module is connected with the input end of the semantic fusion module, the third initial feature map is converted into an optimized feature map based on the semantic fusion module, and the output end of the semantic fusion module is connected with the input end of the feature pyramid network module.

3. The wood surface defect detection system of claim 2, wherein: The sizes of the first initial feature map, the second initial feature map and the third initial feature map are different.

4. The wood surface defect detection system of claim 1, wherein: The basic module is composed of a convolution layer, a batch normalization layer and a basic module composite, and the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are each composed of an efficient layer aggregation network module and a maximum value pooling layer.

5. The wood surface defect detection system of claim 2, wherein: The feature pyramid network module is configured to convert the first initial feature map into a first initial fusion feature map, or convert the second initial feature map into a second initial fusion feature map, or convert the optimized feature map into a third initial fusion feature map. An output end of the feature pyramid network module is connected to an input end of the adaptive structure feature fusion module, so as to convert the first initial fusion feature map into a first optimized fusion feature map, or convert the second initial fusion feature map into a second optimized fusion feature map, or convert the third initial fusion feature map into a third optimized fusion feature map.

6. A method for detecting surface defects of wood, applied to the system for detecting surface defects of wood according to any one of claims 1 to 5, characterized in that: The detection method comprises: providing a wood picture and a trunk module, the wood picture being configured to obtain initial feature maps of different sizes based on the trunk module; providing a semantic fusion module, the smallest initial feature map being selected and the optimized feature map being obtained by using the semantic fusion module; providing a feature pyramid network module, the unselected initial feature map and the optimized feature map being input into the feature pyramid network module to obtain initial fusion feature maps of multiple sizes; providing an adaptive structure feature fusion module, all the initial fusion feature maps being input into the adaptive structure feature fusion module to obtain optimized fusion feature maps of multiple sizes; providing a prediction head module, the optimized fusion feature map being configured to obtain a defect map based on the prediction head module; obtaining a defect result on the defect map and outputting the defect result.

7. The method of wood surface defect detection according to claim 6, wherein: When all the initial fusion feature maps are input into the adaptive structure feature fusion module, the optimized fusion feature map of each size corresponds to all the initial fusion feature maps.

8. The method of wood surface defect detection according to claim 6, wherein: The defect result comprises any one or a combination of a live knot, a dead knot, a resin, a crack, a crack knot, a knot missing, a color difference, a decay, and a pith.

9. A method of training a wood surface defect detection system, applied to the wood surface defect detection system according to any one of claims 1-5, characterized in that: The training method comprises: providing a sample picture, and converting the sample picture into data; cleaning the sample picture, and performing affine transformation data enhancement processing on the cleaned sample picture to obtain a first enhanced picture; performing copy-paste data processing on the first enhanced picture to obtain a second enhanced picture; obtaining a default anchor box of the second enhanced picture, re-calculating the default anchor box to obtain a new anchor box, and performing iterative training after replacing the default anchor box with the new anchor box; setting a maximum number of training, and determining whether the number of this training is greater than a preset number; if not, sequentially performing mosaic data enhancement processing and mixed data enhancement processing on the second enhanced picture to obtain a training picture, and inputting the training picture into the wood surface defect detection system to update the data in the detection system; if yes, directly inputting the second enhanced picture into the wood surface defect detection system to update the data in the detection system.

Images