Wood surface defect detection system, detection method and training method

Abstract

The invention relates to the technical field of wood defect detection, in particular to a wood surface defect detection system, a detection method and a training method. A wood surface defect detection system is used for identifying wood surface defects on a wood picture and comprises a trunk module, a neck module and a feature head module, the trunk module comprises an input end and an output end, and the neck module comprises a semantic fusion module, a feature pyramid network module and a self-adaptive structure feature fusion module. The output end of the trunk module is connected with the input end of the semantic fusion module and the input end of the feature pyramid network module, the output end of the semantic fusion module is connected with the input end of the feature pyramid network module, and the output end of the feature pyramid network module is connected with the input end of the adaptive structure feature fusion module. And the output end of the adaptive structure feature fusion module is connected with the feature head module. The problem that an existing detection model is low in detection speed is solved.

Description

[Technical field]

[0001] The present invention relates to the technical field of wood defect detection, and in particular to a wood surface defect detection system, a detection method and a training method. [Background technology]

[0002] Wood is a natural renewable resource derived from trees, and plays an important role in life and production. However, during the production process, defects such as knots, cracks, discoloration, rot and insect pests will appear on the surface of the wood. These defects destroy the integrity and stability of the wood and affect production efficiency. As one of the quality criteria for wood, accurate identification of defects on the surface of wood is beneficial to production. Manual detection methods have low efficiency, low accuracy and high cost in detecting defects due to factors such as high labor costs, different personal detection standards, high work intensity and poor working environment, which in turn affects the determination of wood quality. Technological advances have led to the emergence of other methods for identifying wood defects, such as nuclear magnetic resonance detection, ultrasound, infrared, laser, etc. However, the detection equipment used in these methods is usually expensive, and there is a problem of slow detection speed when detecting wood based on the above detection equipment, which is not suitable for widespread promotion. [Summary of the invention]

[0003] In order to solve the problem of slow detection speed of the existing detection model, the present invention provides a wood surface defect detection system.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a wood surface defect detection system for identifying wood surface defects on wood pictures, the detection system comprising a trunk module, a neck module and a feature head module;

[0005] The backbone module includes an input end and an output end, the neck module includes a semantic fusion module, a feature pyramid network module and an adaptive structural feature fusion module, the output end of the backbone module is respectively connected to the input end of the semantic fusion module and the input end of the feature pyramid network module, the output end of the semantic fusion module is connected to the input end of the feature pyramid network module, the output end of the feature pyramid network module is connected to the input end of the adaptive structural feature fusion module, and the output end of the adaptive structural feature fusion module is connected to the feature head module;

[0006] The wood image obtains initial feature maps of different sizes based on the trunk module, and the initial feature map obtains the optimized feature map based on the semantic fusion module. Part of the initial feature map and the optimized feature map are passed through the feature pyramid network module to obtain the initial fused feature map, and the initial fused feature map is input into the adaptive structural feature fusion module to obtain the optimized fused feature map. The optimized fused feature map obtains the defect map through the feature head module to identify wood surface defects.

[0007] Preferably, the trunk module includes a composite module, a first feature aggregation module, a second feature aggregation module and a third feature aggregation module connected in sequence, the composite module includes two basic modules and a high-efficiency layer aggregation network module connected in sequence, the input end of one of the basic modules is used to receive the wood picture, and the output end is connected to the input end of another basic module, the output end of the other basic module is connected to the input end of the high-efficiency layer aggregation network module, the output end of the high-efficiency layer aggregation network module is connected to the input end of the first feature aggregation module, and the output ends of the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are respectively connected to the input end of the neck module.

[0008] Preferably, the first feature aggregation module is used to output a first initial feature map, and the output end of the first feature aggregation module is connected to the input end of the second feature aggregation module, the semantic fusion module and the feature pyramid network module respectively;

[0009] The second feature aggregation module is used to output a second initial feature map, and the output end of the second feature aggregation module is connected to the input end of the third feature aggregation module, the semantic fusion module and the feature pyramid network module respectively;

[0010] The third feature aggregation module is used to output a third initial feature map, and the output end of the third feature aggregation module is connected to the input end of the semantic fusion module, the third initial feature map is converted into an optimized feature map based on the semantic fusion module, and the output end of the semantic fusion module is connected to the input end of the feature pyramid network module.

[0011] Preferably, the sizes of the first initial feature map, the second initial feature map and the third initial feature map are different.

[0012] Preferably, the basic module is composed of a convolutional layer, a batch normalization layer and a basic module, and the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are all composed of a high-efficiency layer aggregation network module and a maximum pooling layer.

[0013] Preferably, the feature pyramid network module is used to convert the first initial feature map into a first initial fused feature map, or convert the second initial feature map into a second initial fused feature map, or convert the optimized feature map into a third initial fused feature map;

[0014] The output end of the feature pyramid network module is connected to the input end of the adaptive structural feature fusion module to convert the first initial fused feature map into a first optimized fused feature map, or convert the second initial fused feature map into a second optimized fused feature map, or convert the third initial fused feature map into a third optimized fused feature map.

[0015] In order to solve the above technical problems, the present invention provides another technical solution as follows: a wood surface defect detection method, applied to the above wood surface defect detection system, the detection method comprises:

[0016] Provide wood images and trunk modules, and obtain initial feature maps of different sizes based on the trunk modules.

[0017] Provide a semantic fusion module, select the initial feature map of the minimum size and use the semantic fusion module to obtain the optimized feature map;

[0018] Providing a feature pyramid network module, inputting the unselected initial feature map and the optimized feature map into the feature pyramid network module to obtain initial fused feature maps of multiple sizes;

[0019] Providing an adaptive structural feature fusion module, inputting all initial fused feature maps into the adaptive structural feature fusion module to obtain optimized fused feature maps of multiple sizes;

[0020] Providing a prediction head module, the optimized fusion feature map obtains a defect map based on the prediction head module;

[0021] Get the defect results on the defect map and output the defect results.

[0022] Preferably, when all the initial fused feature maps are input into the adaptive structural feature fusion module, the optimized fused feature maps of each size correspond to all the initial fused feature maps.

[0023] Preferably, the defect results include any one or more combinations of slip knots, dead knots, resin, cracks, crack knots, missing knots, color difference, decay, and pith.

[0024] In order to solve the above technical problems, the present invention provides another technical solution as follows: a training method for a wood surface defect detection system, applied to the above wood surface defect detection system, the training method comprises:

[0025] Provide sample images and convert them into data;

[0026] Cleaning the sample image, and performing affine transformation data enhancement processing on the cleaned sample image to obtain a first enhanced image;

[0027] Copy and paste the first enhanced image to obtain a second enhanced image;

[0028] Obtain a default anchor frame of the second enhanced image, recalculate the default anchor frame to obtain a new anchor frame, use the new anchor frame to replace the default anchor frame, and then perform iterative training;

[0029] Set the maximum number of training times and determine whether the number of training times is greater than the preset number;

[0030] If not, the second enhanced image is sequentially subjected to mosaic data enhancement processing and mixed data enhancement processing to obtain a training image, and the training image is input into the wood surface defect detection system to update the data in the detection system;

[0031] If yes, the second enhanced image is directly input into the wood surface defect detection system to update the data in the detection system.

[0032] Compared with the prior art, the wood surface defect detection system, detection method and training method provided by the present invention have the following beneficial effects:

[0033] 1. A wood surface defect detection system provided by an embodiment of the present invention is used to identify wood surface defects on a wood picture. The detection system includes a trunk module, a neck module and a feature head module;

[0034] The trunk module includes an input end and an output end, the neck module includes a semantic fusion module, a feature pyramid network module and an adaptive structural feature fusion module, the output end of the trunk module is respectively connected to the input end of the semantic fusion module and the input end of the feature pyramid network module, the output end of the semantic fusion module is connected to the input end of the feature pyramid network module, the output end of the feature pyramid network module is connected to the input end of the adaptive structural feature fusion module, and the output end of the adaptive structural feature fusion module is connected to the feature head module;

[0035] The wood image obtains initial feature maps of different sizes based on the trunk module, and the initial feature map obtains the optimized feature map based on the semantic fusion module. Part of the initial feature map and the optimized feature map are passed through the feature pyramid network module to obtain the initial fused feature map. The initial fused feature map is input into the adaptive structural feature fusion module to obtain the optimized fused feature map, and the optimized fused feature map obtains the defect map through the feature head module to identify the surface defects of the wood. It should be understood that the semantic fusion module in the present invention enhances the semantic and detail information in the image by integrating the hierarchical feature map generated in the trunk module. The adaptive structural feature fusion module improves the detection accuracy of structural missing that often appears at the edge of the image. In addition to being able to identify multiple defects and multiple defect combinations, the wood surface defect detection system provided in this embodiment can also be improved in efficiency and accuracy compared to existing image recognition.

[0036] 9. The embodiment of the present invention also provides a wood surface defect detection method, which has the same beneficial effects as the above-mentioned wood surface defect detection system, and will not be described in detail here.

[0037] 10. An embodiment of the present invention also provides a training method for a wood surface defect detection system, which can generate training images to update the data in the wood surface defect detection system to improve the accuracy of the wood surface defect detection system for wood surface defect detection.

Brief Description of the Drawings

[0038] Figure 1 It is a structural schematic diagram of a wood surface defect detection system provided by the first embodiment of the present invention.

[0039] Figure 2 It is a schematic diagram of the basic module structure of the wood surface defect detection system provided by the first embodiment of the present invention.

[0040] Figure 3 It is a schematic diagram of the structure of a feature aggregation module of a wood surface defect detection system provided in the first embodiment of the present invention.

[0041] Figure 4 It is a schematic diagram of the network structure of the PAN module in the prior art.

[0042] Figure 5 It is a schematic diagram of the network structure of the neck module in the wood surface defect detection system of the present invention.

[0043] Figure 6 Schematic diagram of the process of wood surface defect detection method provided by the second embodiment of the present invention

[0044] Figure 7 Schematic diagram of the process of the wood surface defect detection system training method provided by the third embodiment of the present invention

[0045] Description of the accompanying drawings:

[0046] 10. Wood surface defect detection system;

[0047] 1. Trunk module; 2. Neck module; 3. Feature head module;

[0048] 11. Composite module; 12. First feature aggregation module; 13. Second feature aggregation module; 14. Third feature aggregation module; 21. Semantic fusion module; 22. Feature pyramid network module; 23. Adaptive structural feature fusion module;

[0049] 111. Basic module; 112. Efficient layer aggregation network module; 121. Efficient layer aggregation network module; 122. Maximum pooling layer;

[0050] 1111. Convolutional layer; 1112. Batch normalization layer; 1113. Basic module. [Specific implementation method]

[0051] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0053] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present invention.

[0054] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not mean the necessary order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0055] The flow chart and block diagram in the accompanying drawings of the present invention illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is determined based on the functions involved. It should be particularly noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs a specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0056] Wood is a natural renewable resource derived from trees, and plays an important role in life and production. However, during the production process, defects such as knots, cracks, discoloration, rot and insect pests will appear on the surface of the wood. These defects destroy the integrity and stability of the wood and affect production efficiency. As one of the quality criteria for wood, accurate identification of defects on the surface of wood is beneficial to production. Manual detection methods have low efficiency, low accuracy and high cost in detecting defects due to factors such as high labor costs, different personal detection standards, high work intensity and poor working environment, which in turn affects the determination of wood quality. Technological advances have led to the emergence of other methods for identifying wood defects, such as nuclear magnetic resonance detection, ultrasound, infrared, laser, etc. However, the detection equipment used in these methods is usually expensive, and there is a problem of slow detection speed when detecting wood based on the above detection equipment, which is not suitable for widespread promotion.

[0057] The present invention is based on the improvement of the existing image recognition network. The existing image recognition network mainly includes three structures: backbone network (backbone), middle network (neck) and head network (head). Among them:

[0058] The backbone network is the core part of the image recognition network, responsible for extracting basic feature information from the input image. It is usually composed of a deep convolutional neural network (CNN), which can gradually extract image features from low-level to high-level.

[0059] The middle network (Neck) connects the backbone network and the head network, and is usually used to further process and optimize the fused feature map. Its goal is to fuse and enhance the multi-level features extracted from the backbone network, making the feature map more effective in subsequent predictions.

[0060] The head network is responsible for performing tasks such as target detection, classification or regression based on the output features of the intermediate network, and finally generates prediction results. It converts feature maps into final outputs such as detection boxes, categories and confidence levels.

[0061] Usually, the backbone module is usually provided with a module for processing graphics and extracting features, as well as a module for extracting features. Exemplarily, the existing image recognition network model is usually provided with a spatial pyramid pooling module (Spatial Pyramid Pooling Fast, referred to as SPPF). When the image input enters the backbone network, SPPF performs multi-scale pooling on the input feature map and splices the pooled results to obtain a multi-scale feature representation. The obtained feature representation can be directly input into the neck module, and the fused feature map is optimized by its internal path aggregation network module (Path Aggregation Network, referred to as PAN), which can improve the robustness and performance of the network. Usually, after fusion by PAN, it can enter the head network (Head) to output the detection result.

[0062] Although the image recognition network of the prior art can recognize images. However, there are many types of wood surface defects. Exemplarily, the wood surface includes nine types of defects, including slipknots, dead knots, resin, cracks, crack knots, missing knots, color difference, decay, and pith. That is, when it is necessary to recognize a complex wood surface, the existing image recognition network can only recognize a single defect, and it needs to use larger network model parameters, which is costly. In addition, the existing image recognition method also has the problem of inaccurate recognition.

[0063] See also Figure 1 The first embodiment of the present invention provides a wood surface defect detection system 10, which is used to identify wood surface defects on wood images. The detection system includes a backbone module 1 (Backbone), a neck module 2 (Neck) and a feature head module 3 (Head). The backbone module 1 includes an input end and an output end, and the neck module 2 includes a semantic fusion module 21, a feature pyramid network module 22 and an adaptive structural feature fusion module 23. The output end of the backbone module 1 is respectively connected to the input end of the semantic fusion module 21 and the input end of the feature pyramid network module 22, the output end of the semantic fusion module 21 is connected to the input end of the feature pyramid network module 22, the output end of the feature pyramid network module 22 is connected to the input end of the adaptive structural feature fusion module 23, and the output end of the adaptive structural feature fusion module 23 is connected to the feature head module 3;

[0064] It should be understood that the source of the wood picture is to take pictures of wood in reality, and the pictures taken can be single wood or multiple wood. There may be many defects on the wood, such as slipknots, dead knots or cracks. The existing technology is relatively low in both recognition accuracy and recognition efficiency. In this embodiment, a wood surface defect detection system 10 with a new structure is provided. First, the trunk module 1 has an input end, and the wood picture can be input into the trunk module 1 through the input end, and initial feature maps of different sizes can be obtained based on the trunk module 1. It should be noted that the initial feature map has changed in size and content compared with the wood picture. After the initial feature map is fused based on the semantic fusion module 21, an optimized feature map can be obtained. It should be noted that the semantic fusion module 21 (Semantics and Detail Infusion, referred to as SDI) is part of the system. The hierarchical features generated by the network in the trunk module 1 can be divided into low-level features and high-level features. As the network depth increases, the image features gradually become abstract. Low-level features retain more image details, but lack sufficient semantic information and may contain unwanted noise. High-level features contain richer semantic information, but due to the significant reduction in resolution, detailed information (such as target location information) may be lost. The SDI module enhances the semantics and detail information in the image by integrating the hierarchical feature maps generated in the backbone module 1. That is, compared with the initial feature map, the semantics and details in the optimized feature map are enhanced.

[0065] Furthermore, part of the initial feature map and the optimized feature map are passed through the Feature Pyramid Network (FPN) module 22 to obtain an initial fused feature map. It should be noted that FPN is proposed for the multi-scale problem in target detection, and its purpose is to improve the detection performance of small objects through a simple network connection. That is, when the initial feature map and the optimized feature map enter the pyramid network module, the detection capability of small objects in the obtained initial fused feature map will be improved.

[0066] Further, the initial fused feature map is input into the adaptive structure feature fusion module 23 (AdaptiveStructure Feature Fusion, ASFF for short) to obtain the optimized fused feature map. The adaptive structure feature fusion module 23 uses the Hadamard product operation to dynamically select and fuse multiple layers of feature maps, so that the network can capture the multi-scale information of the target. The Hadamard product, also known as the element-by-element product, dot product or Schur product, is a special operation between two matrices. It generates a new matrix, each element in the new matrix is ​​the product of the elements at the corresponding positions in the original two matrices. Specifically, for two matrices A and B of the same shape, their Hadamard product C is defined as C = A × B, where Cij = Aij × Bij, where i and j represent the rows and columns of the matrix, respectively. In the existing network structure, the path aggregation network (the Path Aggregation Network, PAN for short) of the model is often used. PAN is a deep learning network structure, mainly used for image classification and target detection tasks. It can effectively improve the performance and accuracy of the model through the method of multi-scale feature fusion and path aggregation. By introducing top-down and bottom-up path aggregation in PAN, feature maps at different levels can be effectively utilized. Exemplarily, the top-down path aggregation module (PA): starting from the feature map with lower resolution, information is passed up layer by layer through upsampling operations. At each level, the low-resolution feature map is laterally connected and point-by-point added with the high-resolution feature map to integrate information at different scales. The bottom-up path aggregation module (PAM): starting from the high-resolution feature map, information is passed down layer by layer through downsampling operations. Similarly, at each level, the high-resolution feature map is laterally connected and point-by-point added with the low-resolution feature map.

[0067] It should be noted that, in the present embodiment, in the feature fusion part, the top-down path in PAN is replaced by an adaptive structural feature fusion module 23. When the existing PAN module performs network fusion information, the model in the system becomes complicated. The adaptive structural feature fusion module 23 enables the network to capture multi-scale information of the target. In addition, the use of the adaptive structural feature fusion module 23 can improve the mAP by 0.3% without reducing the overall mAP, making the model lightweight and reducing the number of parameters of the model by 10%. It should be noted that mAP (mean Average Precision) is an important indicator for evaluating model performance. In addition, the adaptive structural feature fusion module 23 also improves the detection accuracy of knot missing that often appears at the edge of the image, with an improvement effect of about 5%. That is, compared with the initial fusion feature map, the detection accuracy of the optimized fusion feature map is further improved.

[0068] Further, the optimized fusion feature map obtains the defect map through the feature head module 3. Compared with the initial wood image, the network's recognition ability for wood defects is improved. In addition, the experimental results on the data set show that the mAP of the improved network on the test set is 4.5% higher than that of the original network. The improved network improves the detection accuracy without significantly increasing the model size and inference speed. The improved network improves the ability to extract image features and can identify a variety of wood surface defects. This method can meet the accuracy and real-time requirements in actual production. The differences between various wood surface defects include inter-class differences, intra-class differences and scale differences. Among the differences, small inter-class differences will affect model classification. For example, the size of the knots varies greatly, the length of the cracks varies, and the large scale difference affects the detection accuracy of different sizes of the same defects. By using the SDI module to integrate the different size images generated by the network of the backbone module 1 and using the ASFF module to fuse useful information, sufficient useful information is exchanged between the multi-size feature maps, and the problem of defect size differences is alleviated. The feature extraction capability for dense targets and mixed targets is also improved.

[0069] Optionally, the identification of wood defects in this embodiment includes any one or more combinations of slipknots, dead knots, resin, cracks, crack knots, missing knots, color difference, decay, and pith. It should be understood that compared with the existing recognition system, the wood surface defect detection system 10 provided in this embodiment can not only identify multiple defects and multiple defect combinations, but also improve the efficiency and accuracy compared with the existing image recognition.

[0070] Furthermore, please combine Figure 2 The trunk module 1 includes a composite module 11, a first feature aggregation module 12, a second feature aggregation module 13 and a third feature aggregation module 14 connected in sequence. The composite module 11 includes two basic modules 111 and a high-efficiency layer aggregation network module 112 connected in sequence. The input end of a basic module 111 is used to receive a wood picture, and the output end is connected to the input end of another basic module 111. The output end of the other basic module 111 is connected to the input end of the high-efficiency layer aggregation network module 112. The output end of the high-efficiency layer aggregation network module 112 is connected to the input end of the first feature aggregation module 12. The output ends of the first feature aggregation module 12, the second feature aggregation module 13 and the third feature aggregation module 14 are respectively connected to the input end of the neck module 2.

[0071] It should be noted that the backbone module 1 is the core part of the image recognition network, which is responsible for extracting basic feature information from the input image. The design of the backbone module 1 is particularly important. When the backbone image is input from the input end of the composite module 11, the size of the wood image will change. For example, if the size of the image is "640×640", after passing through the first basic module 111, the size of the image will change to "320×320", and after passing through the second basic module 111, the size of the image will change to 1 / 4 of the initial wood image, that is, the size will change to "160×160". Then the wood image with a size of 1 / 4 will pass through the efficient layer aggregation network module 112 (Elan). Elan improves the efficiency and richness of feature extraction by merging the outputs of multiple convolutional layers.

[0072] Further, please combine Figure 3 , 1 / 4 of the wood images will enter the first feature aggregation module 12 after passing through the efficient layer aggregation network module 112. It should be noted that the first feature aggregation module 12, the second feature aggregation module 13 and the third feature aggregation module 14 are all composed of the efficient layer aggregation network module 121 (Elan) and the maximum pooling layer 122 (Maxpool). The maximum pooling layer 122 is a common pooling operation in convolutional neural networks. It divides the input feature map into regions through a sliding window, selects the maximum value in each region, and outputs the pooled feature map. The maximum pooling operation can effectively reduce the size of the feature map, reduce the amount of calculation and memory usage. In addition, the maximum pooling operation can also select the maximum value in the region, and the maximum pooling retains the significant features of the image and suppresses noise.

[0073] Specifically, the sizes of the first initial feature map, the second initial feature map, and the third initial feature map are different. Every time a 1 / 4 wood image passes through a feature aggregation module, its size will change accordingly. In addition, the output end of each feature aggregation module is the output channel of the initial feature map. It should be understood that when a 1 / 4 wood image is input into the first feature aggregation module 12, the 1 / 4 wood image will change into a first initial feature map with a size of 1 / 8. The first feature aggregation module 12 has three output ends, and the first feature aggregation module 12 is used to output the first initial feature map, wherein the first output end is connected to the input end of the second feature aggregation module 13, the second output end is connected to the input end of the feature pyramid network module 22, and the third output end is connected to the input end of the semantic fusion module 21. The first feature aggregation module 12 allows the first initial feature map with a size of 1 / 8 to enter the second feature aggregation module 13, the feature pyramid network module 22, and the semantic fusion module 21 respectively.

[0074] Specifically, the second feature aggregation module 13 is used to output the second initial feature map, and the output end of the second feature aggregation module 13 is connected to the input end of the third feature aggregation module 14, the semantic fusion module 21 and the feature pyramid network module 22 respectively. It should be understood that when the first initial feature map of 1 / 8 enters the second feature aggregation module 13, it will change into a second initial feature map of 1 / 16 in size, and the second feature aggregation module 13 also has three output ends, wherein the first output end is connected to the input end of the third feature aggregation module 14, the second output end is connected to the input end of the feature pyramid network module 22, and the third output end is connected to the input end of the semantic fusion module 21. The second feature aggregation module 13 allows the first initial feature map of 1 / 16 in size to enter the third feature aggregation module 14, the feature pyramid network module 22 and the semantic fusion module 21 respectively.

[0075] Specifically, the third feature aggregation module 14 is used to output the third initial feature map, and the output end of the third feature aggregation module 14 is connected to the input end of the feature pyramid network module 22, and the third initial feature map is converted into an optimized feature map based on the semantic fusion module 21, and the output end of the semantic fusion module 21 is connected to the input end of the feature pyramid network module 22. It should be understood that when the second initial feature map of 1 / 16 enters the third feature aggregation module 14, it will change into a third initial feature map of 1 / 32 in size, and the third feature aggregation module 14 has only one output end. The third feature aggregation module 14 allows the third initial feature map of 1 / 32 in size to enter the semantic fusion module 21. Furthermore, the third initial feature map entering the semantic fusion module 21 will be processed by the semantic fusion module 21 so that the semantics and details in the map are enhanced and the third initial feature map is converted into an optimized feature map, and the optimized feature map then enters the feature pyramid network module 22.

[0076] It can be understood that after being processed by the trunk module 1 and the neck module 2 in this embodiment, the feature pyramid network module 22 can receive the first initial feature map, the second initial feature map and the optimized feature map. The sizes and contents of the first initial feature map, the second initial feature map and the optimized feature map are different. It should be understood that pictures of different sizes are input into the feature pyramid network module 22 for fusion, which can increase the richness of the pictures.

[0077] It should be noted that after the traditional backbone module 1 processes and aggregates the graphics, it usually enters the spatial pyramid pooling module (Spatial Pyramid Pooling Fast, referred to as SPPF). After the backbone network extracts feature maps of different levels, the SPPF is used to fuse the information of different scales through the efficient pooling method of the highest-level feature map. The SPPF module has achieved excellent performance, but the SPPF module only processes the highest-level features, and the feature fusion is not sufficient. In this implementation, the SPPF module is deleted from the backbone module 1, and a semantic fusion module 21 (Semantics and Detail Infusion, referred to as SDI) is added to the neck module 2. The SDI module can more effectively integrate the hierarchical features generated by the network of the backbone module. Specifically, the SDI module effectively integrates shallow and deep features, and the fused feature map helps to alleviate the impact of scale differences on accuracy. The introduction of this module improves the overall detection of wood defects by 2.1% mAP, and has a significant improvement in the detection accuracy of defects with large scale changes, such as slipknots, cracks, and crack knots, which improve the detection accuracy by 5%, 5%, and 4%, respectively.

[0078] Specifically, the basic module 111 (CBS) is composed of a convolutional layer 1111, a batch normalization layer 1112 (BatchNormalization) and a basic module 1113 (Sigmoid-weighted Linear Unit, referred to as SiLU). It should be understood that LReLU (Leaky Rectified Linear Unit) or ReLU (Rectified Linear Unit) is usually set in the existing basic module 111. LReLU and ReLU are commonly used activation functions and are widely used in neural networks. The activation function ReLU has high computational efficiency and enables the model to converge quickly during the learning process. When the input of the ReLU function is negative or zero, the gradient is zero, and the gradient disappears easily. The activation function LReLU optimizes the defects of ReLU in the negative value area. When the network input is negative, it can also propagate backward, so LReLU can effectively prevent the gradient from disappearing. However, the function curve of LReLU is discontinuous at point 0, and non-differentiability is its disadvantage. This embodiment chooses to delete LReLU or ReLU and chooses to add SiLU. The SiLU activation function is continuously differentiable, and its function graph is smoother than ReLU and LReLU. It avoids the non-differentiable problem of ReLU and has the same performance as LReLU in the negative region. The non-monotonic SiLU activation function can better introduce nonlinear factors, help improve the expressive power of the model, and effectively improve the detection accuracy. It should be noted that in order to better apply this system to the field of wood surface defect detection, the modules within the system are redesigned so that the wood surface defect detection system 10 can solve problems that traditional systems cannot solve.

[0079] Exemplarily, the differences between various wood surface defects include inter-class differences, intra-class differences, and scale differences. Among the differences, small inter-class differences will affect model classification, such as the similar shapes between slipknots and dead knots in wood surface defects, and large intra-class differences will affect the correct judgment of defect types. In the basic module 111 in this implementation, the basic module 1113 (SiLU) is innovatively added, and the SiLU activation function in the basic module 1113 introduces nonlinear factors to effectively alleviate the problems of intra-class differences and inter-class differences.

[0080] Further, the feature pyramid network module 22 is used to convert the first initial feature map into a first initial fused feature map, or convert the second initial feature map into a second initial fused feature map, or convert the optimized feature map into a third initial fused feature map;

[0081] The output end of the feature pyramid network module 22 is connected to the input end of the adaptive structural feature fusion module 23 to convert the first initial fused feature map into the first optimized fused feature map, or convert the second initial fused feature map into the second optimized fused feature map, or convert the third initial fused feature map into the third optimized fused feature map.

[0082] It should be understood that the feature pyramid network module 22 improves the detection performance of small objects through a simple network connection, so that the first initial feature map, the second first initial feature map and the optimized feature map are converted into an initial fusion feature map. Further, the key to improving the detection capability of the model is how to effectively integrate low-level and high-level features. The network of the traditional PAN module adds a bottom-up path. The increase in the path makes the network of the PAN module perform well in the traditional detection system. But this also increases the complexity of the model feature fusion part of the system. The network of the adaptive structural feature fusion module 23 in this embodiment filters the conflicting information in the space through identity scaling and adaptive fusion, reducing the fusion of useless information. The network of the adaptive structural feature fusion module 23 improves the problem of inconsistency between features of different scales in the feature pyramid network, and the number of network parameters is less than that of the network of the PAN module. In the adaptive structural feature fusion module 23, feature maps from different layers are convolved with a kernel size of 1×1 to obtain corresponding weight parameters. New fusion features can be obtained by multiplying and adding the weight parameters of features from different layers.

[0083] Please also read Figure 4 , Figure 4 It is a schematic diagram of the network structure of the traditional PAN module. It can be seen that when the initial fusion feature map is transmitted from the FPN module to the PAN module, the fusion feature map F6 is converted to the fusion feature map P6, the fusion feature map F7 is converted to the fusion feature map P7, and the fusion feature map F8 is converted to the fusion feature map P8. In the process of image feature fusion, the fusion feature map P6 often refers to the features of other elements in addition to the fusion feature map F6. For example, the fusion feature map P6 will refer to the features of the fusion feature map F8, and its network flow path is: the fusion feature map F8 is first converted to the fusion feature map P8, the fusion feature map P8 is then fused into the fusion feature map P7, and the fusion feature map P7 is then fused into the fusion feature map P6. This fusion method is relatively indirect, that is, if the fusion feature map P6 wants to refer to the features of the fusion feature map F8, it must refer to the features of the fusion feature map F8 and the fusion feature map F7. This will increase the complexity of the model feature fusion part and reduce the efficiency of the entire feature fusion.

[0084] Please also read Figure 5 , Figure 5Schematic diagram of the network structure of the neck module 2 in this embodiment. The feature pyramid network module 22 (Feature Pyramid Network, referred to as FPN) and the adaptive structure feature fusion module 23 (Adaptive Structure Feature Fusion, referred to as ASFF) are two different network structures. Exemplarily, after the fusion of the feature pyramid network module 22, the initial feature map B3 is converted to the initial fusion feature map F3, the initial feature map B4 is converted to the initial fusion feature map F4, and the optimized feature map S5 is converted to the initial fusion feature map F5. Further, when the initial fusion feature map F3, the initial fusion feature map F4 and the initial fusion feature map F5 are transmitted to the adaptive structural feature fusion module 23, the optimized fusion feature map A3 will directly refer to the initial fusion feature map F3, the initial fusion feature map F4 and the initial fusion feature map F5 for fusion, the optimized fusion feature map A4 will directly refer to the initial fusion feature map F3, the initial fusion feature map F4 and the initial fusion feature map F5 for fusion, and the optimized fusion feature map A5 will directly refer to the initial fusion feature map F3, the initial fusion feature map F4 and the initial fusion feature map F5 for fusion, that is, the biggest difference between the adaptive structural feature fusion module 23 and the traditional PAN module is that the adaptive structural feature fusion module 23 will adaptively select fusion, and the fusion method is more direct and efficient. Specifically, the use of the adaptive structural feature fusion module 23 can improve the mAP by 0.3% without reducing the overall mAP, making the model lightweight and reducing the number of parameters of the model by 10%. In addition, the detection accuracy of missing knots at the edge of the image can be improved by about 5%.

[0085] It should be noted that there is a corresponding relationship between the sizes of the initial feature map, the optimized feature map, the initial fused feature map, and the optimized fused feature map. Exemplarily, the network architectures of the initial feature map B3, the initial fused feature map F3, and the optimized fused feature map A3 are different, but the sizes of their pictures are the same. Similarly, the network architectures of the initial feature map B4, the initial fused feature map F4, and the optimized fused feature map A4 are different, but the sizes of their pictures are the same. I will not go into too much detail here.

[0086] The second embodiment of the present invention further provides a wood surface defect detection method, which is applied to the above-mentioned wood surface defect detection system. The detection method includes:

[0087] S1, providing a wood image and a trunk module, and obtaining initial feature maps of different sizes based on the trunk module;

[0088] S2, provides a semantic fusion module, selects the initial feature map of the minimum size and uses the semantic fusion module to obtain the optimized feature map;

[0089] S3, providing a feature pyramid network module, inputting the unselected initial feature map and the optimized feature map into the feature pyramid network module to obtain initial fused feature maps of multiple sizes;

[0090] S4, providing an adaptive structural feature fusion module, inputting all initial fused feature maps into the adaptive structural feature fusion module to obtain optimized fused feature maps of multiple sizes;

[0091] S5, providing a prediction head module, optimizing the fusion feature map to obtain a defect map based on the prediction head module;

[0092] S6, obtaining defect results on the defect map and outputting the defect results.

[0093] It can be understood that in this embodiment, the wood needs to be photographed first, and after the photographing is completed, the photographed wood picture can be input into the wood surface defect detection system, the wood picture can be input into the trunk module, and initial feature maps of different sizes can be obtained based on the trunk module. Exemplarily, if the size of the wood picture is "640×640", there are three sizes of initial feature maps, and the sizes of the initial feature maps can be: "80×80", "40×40" and "20×20" respectively.

[0094] Further, the feature map with the smallest size in the initial feature map is selected, and the optimized feature map is obtained by using the semantic fusion module. Exemplarily, the size of "20×20" among the above three initial feature maps is the smallest, so the initial feature map of "20×20" will be converted into the optimized feature map through the semantic fusion module. Further, the unselected initial feature map and the optimized feature map are input into the feature pyramid network module. That is, the initial feature maps of size "80×80", "40×40" and the optimized feature map of size "20×20" will be input into the feature pyramid network module and converted into initial fused feature maps of multiple sizes. It should be noted that when converted into the initial fused feature map, the size of the initial fused feature map corresponds to the unconverted initial feature map or the optimized feature map. It should be understood that the initial feature map of "80×80" will be converted into the initial fused feature map of "80×80", and the optimized feature map of "20×20" will be converted into the initial fused feature map of "20×20". It should be understood that the detection capability of small objects in the initial fused feature map will be improved.

[0095] Further, all the initial fused feature maps are input into the adaptive structural feature fusion module to obtain optimized fused feature maps of multiple sizes. It should be understood that the detection accuracy of the optimized fused feature map is further improved compared to the initial fused feature map.

[0096] It should be noted that when all the initial fused feature maps are input into the adaptive structural feature fusion module, the optimized fused feature maps of each size correspond to all the initial fused feature maps. For example, the initial fused feature map of "80×80" will be converted to the optimized fused feature map of "80×80", and the initial fused feature map of "20×20" will be converted to the optimized fused feature map of "20×20".

[0097] Further, a prediction head module is provided, and the optimized fusion feature map is used to obtain a defect map based on the prediction head module. Compared with the initial wood image, the network will find it difficult to identify and detect wood defects on the wood image, and when the wood image is converted into a defect map, the network will quickly and accurately obtain the defect results on the defect map and output the defect results. Optionally, the defect results include any one or more combinations of live knots, dead knots, resin, cracks, crack knots, missing knots, color difference, decay, and pith.

[0098] It should be understood that the existing technology detects few types of defects or only detects a single type, or uses a large number of network model parameters, or uses data collected in an ideal environment for training, which does not conform to the actual situation in industrial production where the defect types are complex and diverse and the detection environment is complex. The data set used in this method is collected in an actual industrial production environment. It obtains a large amount of real data from the production line and can detect live knots, dead knots, resins, cracks, crack knots, missing knots, color differences, decay, and pith. A sufficient number of data sets are obtained using data enhancement technology, and the number of various defects in the data set is balanced. The use of data enhancement technology avoids the cost increase caused by collecting, cleaning, and annotating image data in actual applications. At the same time, this technology can help the network better understand the differences in data, enhance the recognition ability of a few defects, improve the generalization ability of the model, and reduce the overfitting of the network to the training data. After using data enhancement technology, the network's ability to recognize defects is improved.

[0099] The third embodiment of the present invention further provides a training method for a wood surface defect detection system, which is applied to the wood surface defect detection system. The training method includes:

[0100] A1, provide sample images and clean them;

[0101] A2, performing affine transformation data enhancement processing on the cleaned sample image to obtain a first enhanced image;

[0102] A3, copying and pasting the first enhanced image to obtain a second enhanced image;

[0103] A4, obtaining a default anchor frame of the second enhanced image, recalculating the default anchor frame to obtain a new anchor frame, using the new anchor frame to replace the default anchor frame and then performing iterative training;

[0104] A5, setting the maximum number of iterative training times, and judging whether the number of this training times is greater than the preset number;

[0105] A6, if not, sequentially performing mosaic data enhancement processing and mixed data enhancement processing on the second enhanced image to obtain a training image, and inputting the training image into the wood surface defect detection system to update the data in the detection system;

[0106] A7, if so, directly input the second enhanced image into the wood surface defect detection system to update the data in the detection system.

[0107] Understandably, the traditional data training method usually processes the input sample images and then directly feeds the processed images into the system. The problem with this method is that the sample images may be different from the images of trees in the real world, that is, the sample images used in training the model do not fit the actual distribution of trees in reality.

[0108] In order to solve this technical problem. In this embodiment, the sample image is first cleaned. Exemplarily, the method for cleaning the sample image is as follows: correcting the image labels in the public wood surface defect dataset, cropping the black edges of the dataset image, and converting the TXT tag file into an XML file in VOC format. It should be understood that when the network reads the wood surface defects in the image, it will have requirements for the format of the image. Converting TXT to XML file format is more in line with the network in the system to read and identify the defects on the image.

[0109] Further, the cleaned sample image is subjected to affine transformation data enhancement processing to obtain a first enhanced image. It should be understood that the sample image itself may have problems such as unclear brightness and crooked images. In this embodiment, the affine transformation data enhancement processing is mainly to adjust the rotation of the internal graphics of the sample image, the brightness of the graphics, and whether to add noise points to the graphics, that is, the main function of the data enhancement processing is to enrich the content of the sample image. After the affine transformation data enhancement processing, the anti-interference and robustness of the image are enhanced, making the image more realistic.

[0110] Further, the first enhanced image is processed by copying and pasting data to obtain a second enhanced image. It should be understood that the frequencies of defects on the wood image are different, for example, some defects appear more frequently and some appear less frequently. In the process of pattern recognition, designers often hope to balance the phenomenon of different frequencies of occurrence of these defects. When training the model, defects with more frequencies will result in too much training, while defects with less frequencies will result in too little training. After copying and pasting data processing, the frequencies of occurrence of different defects will be balanced, so that different defects can be trained in a balanced manner in the future.

[0111] Furthermore, a new anchor frame is reset, and the new anchor frame is used to replace the default anchor frame. It should be understood that the size of the sample image is usually different from the image size adapted by the network, so the anchor frame of the image needs to be reset. Exemplarily, the size of the sample image is usually "2800*1440" and the image size adapted by the network is "640*640". If the size of the sample image of "2800*1440" is directly used for training, it will affect the training speed of the system. It should be understood that resetting the anchor frame will greatly increase the training speed.

[0112] Specifically, the purpose of iterative training is to make the trained pictures more consistent with the defect distribution of wood in the real world. Exemplarily, iterative training can be to select the Adam (Adaptive Moment Estimation) optimizer, with an initial learning rate of 0.001, and use the cosine annealing method to update the learning rate, a training epoch of 120, and a batch size of 16. Among them, the training epoch refers to a complete cycle of forward propagation and back propagation of the entire training data set on the neural network. Batch Size: Batch size refers to the number of samples processed by the model in one forward and backward propagation. The choice of batch size will affect the stability and speed of training. It should be noted that Adam is an efficient optimization algorithm used for parameter updates in deep learning, which aims to adaptively adjust the learning rate of each parameter, thereby improving the convergence speed and generalization ability of the model.

[0113] Further, determine whether the number of times of this training is greater than the preset number of times. Exemplarily, assume that the preset number of times is 70% of the maximum number of iterations. That is, when 70% of the iterations are completed, the mosaic data enhancement processing and the mixed (Mixup) data enhancement processing are turned off after the training round. It should be understood that the mosaic data enhancement processing is to randomly crop four pictures and splice them into a new picture as training data. The mosaic data enhancement processing increases the diversity of the data and enriches the background information of the picture. The mixed data enhancement processing is to mix two random samples in proportion. The mixed data enhancement processing will not cause unnatural situations after the images are mixed.

[0114] It should be understood that if the number of training times is less than the preset number, the second enhanced image needs to undergo mosaic data enhancement processing and mixed data enhancement processing to obtain a training image, and then the training image is input into the wood surface defect detection system to update the data in the detection system.

[0115] If the number of training times is greater than the preset number, there is no need to go through mosaic data enhancement processing and mixed data enhancement processing. The second enhanced image can be directly input into the wood surface defect detection system to update the data in the detection system, which greatly speeds up the training efficiency.

[0116] It can be understood that the training method of the wood surface defect detection system provided in this embodiment can generate training images to update the data in the wood surface defect detection system to improve the accuracy of the wood surface defect detection system for wood surface defect detection.

[0117] The above is a detailed introduction to a wood surface defect detection system, detection method and training method disclosed in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention, and any modifications, equivalent substitutions and improvements made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A wood surface defect detection system for identifying wood surface defects on wood images, characterized in that: The detection system includes a trunk module, a neck module and a feature head module; The backbone module includes an input end and an output end, the neck module includes a semantic fusion module, a feature pyramid network module and an adaptive structural feature fusion module, the output end of the backbone module is respectively connected to the input end of the semantic fusion module and the input end of the feature pyramid network module, the output end of the semantic fusion module is connected to the input end of the feature pyramid network module, the output end of the feature pyramid network module is connected to the input end of the adaptive structural feature fusion module, and the output end of the adaptive structural feature fusion module is connected to the feature head module; The wood image obtains initial feature maps of different sizes based on the trunk module, and the initial feature map obtains the optimized feature map based on the semantic fusion module. Part of the initial feature map and the optimized feature map are passed through the feature pyramid network module to obtain the initial fused feature map, and the initial fused feature map is input into the adaptive structural feature fusion module to obtain the optimized fused feature map. The optimized fused feature map obtains the defect map through the feature head module to identify wood surface defects.

2. The wood surface defect detection system according to claim 1, characterized in that: The trunk module includes a composite module, a first feature aggregation module, a second feature aggregation module and a third feature aggregation module connected in sequence; the composite module includes two basic modules and a high-efficiency layer aggregation network module connected in sequence; the input end of one basic module is used to receive the wood picture, and the output end is connected to the input end of another basic module; the output end of the other basic module is connected to the input end of the high-efficiency layer aggregation network module; the output end of the high-efficiency layer aggregation network module is connected to the input end of the first feature aggregation module; the output ends of the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are respectively connected to the input end of the neck module.

3. The wood surface defect detection system according to claim 1, characterized in that: The first feature aggregation module is used to output a first initial feature map, and the output end of the first feature aggregation module is connected to the input end of the second feature aggregation module, the semantic fusion module and the feature pyramid network module respectively; The second feature aggregation module is used to output a second initial feature map, and the output end of the second feature aggregation module is connected to the input end of the third feature aggregation module, the semantic fusion module and the feature pyramid network module respectively; The third feature aggregation module is used to output a third initial feature map, and the output end of the third feature aggregation module is connected to the input end of the semantic fusion module, the third initial feature map is converted into an optimized feature map based on the semantic fusion module, and the output end of the semantic fusion module is connected to the input end of the feature pyramid network module.

4. The wood surface defect detection system according to claim 1, characterized in that: The sizes of the first initial feature map, the second initial feature map and the third initial feature map are different.

5. The wood surface defect detection system according to claim 1, characterized in that: The basic module is composed of a convolutional layer, a batch normalization layer and a basic module, and the first feature aggregation module, the second feature aggregation module and the third feature aggregation module are all composed of an efficient layer aggregation network module and a maximum pooling layer.

6. The wood surface defect detection system according to claim 1, characterized in that: The feature pyramid network module is used to convert the first initial feature map into a first initial fused feature map, or convert the second initial feature map into a second initial fused feature map, or convert the optimized feature map into a third initial fused feature map; The output end of the feature pyramid network module is connected to the input end of the adaptive structural feature fusion module to convert the first initial fused feature map into a first optimized fused feature map, or convert the second initial fused feature map into a second optimized fused feature map, or convert the third initial fused feature map into a third optimized fused feature map.

7. A wood surface defect detection method, applied to the wood surface defect detection system according to any one of claims 1 to 6, characterized in that: The detection method comprises: Provide wood images and trunk modules, and obtain initial feature maps of different sizes based on the trunk modules. Provide a semantic fusion module, select the initial feature map of the minimum size and use the semantic fusion module to obtain the optimized feature map; Providing a feature pyramid network module, inputting the unselected initial feature map and the optimized feature map into the feature pyramid network module to obtain initial fused feature maps of multiple sizes; Providing an adaptive structural feature fusion module, inputting all initial fused feature maps into the adaptive structural feature fusion module to obtain optimized fused feature maps of multiple sizes; Providing a prediction head module, the optimized fusion feature map obtains a defect map based on the prediction head module; Get the defect results on the defect map and output the defect results.

8. The method for detecting wood surface defects according to claim 1, wherein: When all the initial fused feature maps are input into the adaptive structure feature fusion module, the optimized fused feature maps of each size correspond to all the initial fused feature maps.

9. The wood surface defect detection method according to claim 1, characterized in that: The defect results include any one or more combinations of slip knots, dead knots, resin, cracks, crack knots, missing knots, color difference, decay, and pith.

10. A training method for a wood surface defect detection system, applied to the wood surface defect detection system as claimed in any one of claims 1 to 6, characterized in that: The training method comprises: Provide sample images and convert them into data; Cleaning the sample image, and performing affine transformation data enhancement processing on the cleaned sample image to obtain a first enhanced image; Copy and paste the first enhanced image to obtain a second enhanced image; Obtain a default anchor frame of the second enhanced image, recalculate the default anchor frame to obtain a new anchor frame, use the new anchor frame to replace the default anchor frame, and then perform iterative training; Set the maximum number of training times and determine whether the number of training times is greater than the preset number; If not, the second enhanced image is sequentially subjected to mosaic data enhancement processing and mixed data enhancement processing to obtain a training image, and the training image is input into the wood surface defect detection system to update the data in the detection system; If yes, the second enhanced image is directly input into the wood surface defect detection system to update the data in the detection system.

Images